Computer processes behind an enhanced version of aqua

ABSTRACT

The invention relates, in part, to systems and methods for scoring a sample containing tumor tissue from a cancer patient. The score obtained from these methods can be indicative of a likelihood that a patient may respond positively to immunotherapy. The invention also relates, in part, to methods of deriving a value for % biomarker positivity (PBP) for all cells or optionally, one or more subsets thereof, present in a field of view of a tissue sample from a cancer patient. The values for PBP can be indicative of a patient&#39;s response to immunotherapy.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is the U.S. National Stage of International PatentApplication No. PCT/US2016/058280, filed Oct. 21, 2016, which claims thebenefit of priority from U.S. Provisional Patent Application No.62/245,938, filed Oct. 23, 2015, U.S. Provisional Patent Application No.62/259,328, filed Nov. 24, 2015, and U.S. Provisional Patent ApplicationNo. 62/301,037, filed Feb. 29, 2016. The contents of these applicationsare incorporated herein by reference in their entirety.

BACKGROUND

The present invention relates generally to the field of cancertreatment.

SUMMARY OF THE INVENTION

Disclosed herein, in one aspect, are imaging systems comprising an imageprocessing controller comprising a processing circuit configured toexecute instructions stored on a computer-readable medium which causethe controller of the image processing system to:

configured to receive image data from an imaging device, the image datadescribing a sample of tumor tissue taken from a cancer patient; anddetermine, using the image data, a score representative of a nearnessbetween at least one pair of cells in the tumor tissue, a first memberof the at least one pair of cells expressing a first biomarker and asecond member of the at least one pair of cells expressing a secondbiomarker that is different from the first biomarker; wherein the scoreis indicative of a likelihood that the cancer patient will respondpositively to immunotherapy. In some embodiments, the scorerepresentative of a nearness between at least one pair cells isrepresentative of an extent that the pair of cells are within apredetermined proximity of one another. In some embodiments, thenearness is assessed on a pixel scale. In some embodiments, thepredetermined proximity between the pair of cells ranges from about 1pixel to about 100 pixels. In some embodiments, the predeterminedproximity between the pair of cells ranges from about 5 pixel to about40 pixels. In some embodiments, the predetermined proximity between thepair of cells ranges from about 0.5 μm to about 50 μm. In someembodiments, the predetermined proximity between the pair of cellsranges from about 2.5 μm to about 20 μm. In some embodiments, the scoreis determined by obtaining a proximity between the boundaries of thepair of cells. In some embodiments, the score is determined by obtaininga proximity between the centers of mass of the pair of cells. In someembodiments, the score is determined using boundary logic based on aperimeter around a selected first cell of the pair of cells. In someembodiments, the score is determined by determining an intersection inthe boundaries of the pair of cells. In some embodiments, the score isdetermined by determining an area of overlap of the pair of cells. Insome embodiments, the image processing controller is further configuredto separate the image data into unmixed image data; and provide the datathrough a plurality of data channels, in which the unmixed image data ina first data channel describes fluorescence signals attributable to thefirst biomarker and the unmixed image data in a second data channeldescribes fluorescence signals attributable to the second biomarker. Insome embodiments, to determine the score, the image processingcontroller is configured to dilate fluorescence signals attributable tothe first biomarker from the first data channel by a predeterminedmargin that is selected to encompass proximally located cells expressingthe second biomarker to generate a dilated first biomarker mask;determine an interaction area, wherein the interaction area is a firsttotal area for all cells which express the second biomarker and areencompassed within the dilated fluorescence signals attributable to thecells expressing the first biomarker; and divide the interaction area bya normalization factor, and multiplying the resulting quotient by apredetermined factor to arrive at a spatial proximity score. In someembodiments, the normalization factor is a total area for all cells thathave a capacity to express the second biomarker. In some embodiments,the interaction area is determined by combining the dilated firstbiomarker mask with a mask representative of cells that express thesecond biomarker, determined from signals of the second data channel. Insome embodiments, a third data channel describes fluorescence signalsattributable to cell nuclei and a fourth data channel describesfluorescence signals attributable to tumor area in the sample. In someembodiments, the total area for all cells that have a capacity toexpress the second biomarker is determined by combining a cell maskrepresentative of all cells in the sample, based on signals from thethird data channel, and a tumor area mask representative of the tumorarea on the sample, based on signals from the fourth data channel. Insome embodiments, combining the cell mask and the tumor area maskcomprises removing the tumor area mask from the cell mask. In someembodiments, each mask is generated by assigning a binary value to eachpixel of image data from a selected channel. In some embodiments, thebinary value is assigned to each value by a threshold function, whereinthe threshold function assigns a value of 1 to each pixel that has anintensity equal to or greater than a predetermined intensity threshold,and assigns a value of 0 to each pixel that has an intensity below thepredetermined intensity threshold. In some embodiments, the binary valueis assigned to each value by a histogram threshold function, wherein thehistogram threshold function uses a sliding scale of pixel intensity todetermine a threshold, and assigns a value of 1 to each pixel that hasan intensity equal to or greater than an intensity threshold, andassigns a value of 0 to each pixel that has an intensity below theintensity threshold. In some embodiments, the threshold is determined bysumming an intensity of every pixel into a total intensity; multiplyinga threshold percentage by the total intensity to obtain a cut-off sum;grouping all pixels by intensity in a histogram; and summing the pixelintensities from lowest to highest until the cut-off sum is achieved;wherein the last highest pixel intensity visited in the process is theintensity threshold. In some embodiments, combining the masks isperformed using an “and” operation; wherein the “and” operation is amultiplication of the pixel intensity of a pixel in a first mask by thepixel intensity of the pixel in a second mask. In some embodiments,combining the masks is performed using an “out” operation; wherein the“out” operation removes the a second mask from a first mask. In someembodiments, the image processing controller is further configured toobtain image data at a low magnification representative of theconcentration of the first or the second biomarker in the image;identify areas that include the highest concentration of the first orthe second biomarker; select a predetermined number of the areasincluding the highest concentration of the first or the secondbiomarker; send instructions to imaging device to obtain highmagnification image data for the predetermined number of areas; andwherein the high magnification image data is provided to the controllerto be analyzed and used to determine the score. In some embodiments, thelow magnification is less than or equal to 10× magnification and whereinthe high magnification is greater than 10×. In some embodiments, the lowmagnification is 10× magnification and wherein the high magnification is40×. In some embodiments, the low magnification is 10× magnification andwherein the high magnification is 20×. In some embodiments, the lowmagnification is 4× magnification and wherein the high magnification is40×. In some embodiments, the low magnification is 4× magnification andwherein the high magnification is 20×. In some embodiments, the imageprocessing unit is further configured to associate the score withmetadata associated with the images of the sample. In some embodiments,the image processing unit generates a report including the score. Insome embodiments, the image processing unit is further configured toprovide the score to an operator to determine immunotherapy strategy. Insome embodiments, the image processing unit is further configured torecord the score in a database. In some embodiments, the imageprocessing unit is further configured to associate the score with apatient's medical record. In some embodiments, the image processing unitis further configured to display the score on a display device. In someembodiments, the first member of the at least one pair of cellscomprises a tumor cell and the second member of the at least one pair ofcells comprises a non-tumor cell. In some embodiments, the non-tumorcell comprises an immune cell. In some embodiments, the first and secondmembers of the at least one pair of cells comprise immune cells. In someembodiments, the first member of the at least one pair of cellscomprises a tumor cell, a myeloid cell, or a stromal cell and the secondmember of the at least one pair of cells comprises an immune cell. Insome embodiments, the tumor cell, myeloid cell, or stromal cellexpresses PD-L1 and the immune cell expresses PD-1. In some embodiments,the image processing unit is further configured to select apredetermined number of fields of view available from the samplecomprising tumor tissue taken from the cancer patient and determine thescore from data associated with each field of view. In some embodiments,the sample is stained with a plurality of fluorescence tags, and whereineach of the fluorescence tags is directed to a specific biomarker. Insome embodiments, the plurality of fluorescence tags comprises a firstfluorescence tag for the first biomarker and a second fluorescence tagfor the second biomarker. In some embodiments, the margin ranges fromabout 1 to about 100 pixels. In some embodiments, the proximally locatedcells expressing the second biomarker are within about 0.5 to about 50μm of a plasma membrane of the cells that express the first biomarker.In some embodiments, the predetermined factor is 10⁴. In someembodiments, the first member of the at least one pair of cellsexpresses a first biomarker selected from the group consisting of PD-L1,PD-L2, B7-H3, B7-H4, HLA-DR, Galectin 9, CD80, CD86, 4.1BBL, ICOSL,CD40, OX40L, IDO-1, GITRL, and combinations thereof, and the secondmember of the at least one pair of cells expresses a second biomarkerselected from the group consisting of PD-1, TIM3, LAG3, 41BB, OX40,CTLA-4, CD40L, CD28, GITR, ICOS, CD28, and combinations thereof. In someembodiments, the threshold value ranges from about 500 to about 5000. Insome embodiments, the threshold value is about 900 plus or minus 100. Insome embodiments, the immunotherapy comprises immune checkpoint therapy.In some embodiments, the method provides a superior predictive powercompared to quantitation of expression of the first biomarker orquantitation of expression of the second biomarker. In some embodiments,the predictive power is quantified as a positive predictive value, anegative predictive value, or a combination thereof. In someembodiments, the positive predictive value is 65% or greater. In someembodiments, the positive predictive value is 70% or greater. In someembodiments, the positive predictive value is 75% or greater. In someembodiments, the negative predictive value is 65% or greater. In someembodiments, the negative predictive value is 80% or greater.

In another aspect, disclosed herein are systems for determining a scorerepresentative of a proximity between at least one pair of cells in atumor tissue sample, the system comprising a memory, the memorycomprising a spectral unmixer receiving image data from an imagingdevice, separating the image data into unmixed image data, and providingthe data through a plurality of data channels, in which the unmixedimage data in a first data channel describes fluorescence signalsattributable to a cell nucleus, the unmixed data in a second datachannel describes fluorescence signals attributable to tumor tissue, theunmixed data in a third data channel describes fluorescence signalsattributable to a first biomarker, and the unmixed image data in afourth data channel describes fluorescence signals attributable to asecond biomarker; a cell masker using image data from the first datachannel and generating an image representative of all nuclei in a fieldof view and dilating the image to generate a cell mask; a tumor areamasker using image data from the second data channel and generating amask of all tumor area in the field of view; a first biomarker maskerusing image data from the third data channel and generating a firstbiomarker mask of all cells in the field of view that express the firstbiomarker; a second biomarker masker using image data from the fourthdata channel and generating a second biomarker mask of all cells in thefield of view that express the second biomarker; a tumor maskercombining the cell mask and the tumor area mask to generate at least oneof a non-tumor cell mask and a tumor cell mask; a dilator dilating thefirst biomarker mask to generate a dilated first biomarker mask; aninteraction masker for combining the dilated first biomarker mask andthe second biomarker mask to generate an interaction mask; aninteraction calculator calculating a spatial proximity score. In someembodiments, the interaction calculator divides the area of theinteraction mask by the area of one of the non-tumor cell mask, thetumor cell mask, and the cell mask. In some embodiments, the spatialproximity score is calculated according to the equation:

${SPS} = {\frac{A_{I}}{A_{C}} \times 10^{4}}$wherein A_(I) is a total interaction area (area of the interaction mask)and A_(C) is the total area of the at least one of the non-tumor cellmask, the tumor cell mask, and the cell mask. In some embodiments, themethod provides a superior predictive power compared to quantitation ofexpression of the first biomarker or quantitation of expression of thesecond biomarker. In some embodiments, the predictive power isquantified as a positive predictive value, a negative predictive value,or a combination thereof. In some embodiments, the positive predictivevalue is 65% or greater. In some embodiments, the positive predictivevalue is 70% or greater. In some embodiments, the positive predictivevalue is 75% or greater. In some embodiments, the negative predictivevalue is 65% or greater. In some embodiments, the negative predictivevalue is 80% or greater.

In another aspect, disclosed herein are methods of processing an imageof a tissue sample to determine a score representative of a proximitybetween at least one pair of cells in the tissue, the method comprising:receiving image data associated with a tissue sample comprising tumortissue taken from a cancer patient; selecting at least one field of viewfrom the sample comprising tumor tissue taken from the cancer patient,which is stained with a plurality of fluorescence tags, which selectionis biased toward selecting at least one field of view that contains agreater number of cells that express the first biomarker relative toother fields of view; dilating, for each of the selected fields of view,fluorescence signals attributable to the first biomarker by a marginsufficient to encompass proximally located cells expressing the secondbiomarker; dividing a first total area for all cells from each of theselected fields of view, which express the second biomarker and areencompassed within the dilated fluorescence signals attributable to thecells expressing the first biomarker, with a normalization factor, andmultiplying the resulting quotient by a predetermined factor to arriveat a spatial proximity score; and recording the score, which score whencompared to a threshold value is indicative of a likelihood that thecancer patient will respond positively to immunotherapy. In someembodiments, the method provides a superior predictive power compared toquantitation of expression of the first biomarker or quantitation ofexpression of the second biomarker. In some embodiments, the predictivepower is quantified as a positive predictive value, a negativepredictive value, or a combination thereof. In some embodiments, thepositive predictive value is 65% or greater. In some embodiments, thepositive predictive value is 70% or greater. In some embodiments, thepositive predictive value is 75% or greater. In some embodiments, thenegative predictive value is 65% or greater. In some embodiments, thenegative predictive value is 80% or greater.

In another aspect, disclosed herein are systems for determining a valueof biomarker positivity in a tissue sample, the system comprising aprocessing circuit comprising a memory, the memory comprising a spectralunmixer receiving image data from an imaging device, separating theimage data into unmixed image data, and providing the data through aplurality of data channels, in which the unmixed image data in a firstdata channel describes fluorescence signals attributable to a cellnucleus, the unmixed data in a second data channel describesfluorescence signals attributable to a subset tissue of interest, andthe unmixed data in a third data channel describes fluorescence signalsattributable to a first biomarker; a cell masker using image data fromthe first data channel and generating an image representative of allnuclei in a field of view and dilating the image to generate a cellmask; a subset area masker using image data from the second data channeland generating a mask of all subset area in the field of view; a firstbiomarker masker using image data from the third data channel andgenerating a first biomarker mask of all cells in the field of view thatexpress the first biomarker; a subset masker combining the cell mask andthe subset area mask to generate at least one of a non-subset cell maskand a subset cell mask; an combination masker for combining thebiomarker mask and at least one of the non-subset cell mask and thesubset cell mask; a positivity calculator for calculating a biomarkerpositivity value. In some embodiments, the calculator divides an area ofthe biomarker mask by an area of the at least one of the non-subset cellmask and the subset cell mask. In some embodiments, the subset tissue ofinterest is tumor tissue, the subset cell is a tumor cell, and thenon-subset cell is a non-tumor cell.

In some embodiments, the first biomarker comprises a biomarker selectedfrom PD-L1, PD-L2, B7-H3, B7-H4, HLA-DR, Galectin 9, CD80, CD86, 4.1BBL,ICOSL, CD40, OX40L, GITRL, PD-1, TIM3, LAG3, 41BB, OX40, CTLA-4, CD40L,CD28, GITR, ICOS, CD28, CD3, CD4, CD8, FoxP3, CD25, CD16, CD56, CD68,CD163, CD80, and CD86.

In some embodiments, the second biomarker is different from the firstbiomarker and comprises a biomarker selected from PD-L1, PD-L2, B7-H3,B7-H4, HLA-DR, Galectin 9, CD80, CD86, 4.1BBL, ICOSL, CD40, OX40L,IDO-1, GITRL, PD-1, TIM3, LAG3, 41BB, OX40, CTLA-4, CD40L, CD28, GITR,ICOS, CD28, CD3, CD4, CD8, FoxP3, CD25, CD16, CD56, CD68, CD163, CD80,and CD86

In another aspect, disclosed herein are systems comprising an imageprocessing unit configured to receive image data from an imaging device,the image data describing a sample of tissue taken from a patient; anddetermine a biomarker positivity value representative of an amount ofcells of a type in the sample expressing a biomarker of interestrelative to the total amount of cells of the type in the sample.

Disclosed herein, in one aspect, are imaging systems for scoring asample comprising tumor tissue taken from a cancer patient, the imagingsystem comprising an imaging device comprising a stage for positioningthe sample in an imaging field, an electromagnetic radiation source fordirecting electromagnetic radiation at the sample, and a detectorconfigured to detect electromagnetic radiation from the sample, and acontroller. The controller comprises a user interface for exchanginginformation between an operator and the controller; and a processingcircuit configured to execute instructions stored on a computer-readablemedium. The instructions are configured to cause the controller to: (i)receive information about the detected electromagnetic radiation fromthe imaging device; (ii) generate image data based on the detectedelectromagnetic radiation; (iii) analyze the image data to determine ascore representative of a nearness between at least one pair of cells, afirst member of the least one pair of cells expressing a first biomarkerand a second member of the at least one pair of cells expressing asecond biomarker that is different from the first biomarker; and (iv)record the score, which score when compared to a threshold value isindicative of a likelihood that the cancer patient will respondpositively to immunotherapy.

In some embodiments, the score is representative of a nearness betweenat least one pair cells is representative of an extent that the pair ofcells are within a predetermined proximity of one another. In someembodiments, the nearness is assessed on a pixel scale. In someembodiments, the predetermined proximity between the pair of cellsranges from about 1 pixel to about 100 pixels. In some embodiments, thepredetermined proximity between the pair of cells ranges from about 5pixel to about 40 pixels. In some embodiments, the predeterminedproximity between the pair of cells ranges from about 0.5 μm to about 50μm. In some embodiments, the predetermined proximity between the pair ofcells ranges from about 2.5 μm to about 20 μm.

In some embodiments, the score is calculated by obtaining a proximitybetween the boundaries of the pair of cells. In some embodiments, thescore is calculated by obtaining a proximity between the centers of massof the pair of cells. In some embodiments, the score is calculated usingboundary logic based on a perimeter around a selected first cell of thepair of cells. In some embodiments, the score is calculated bydetermining an intersection in the boundaries of the pair of cells. Insome embodiments, the score is calculated by determining an area ofoverlap of the pair of cells.

In some embodiments, generating the image data comprises: (i) separatingthe information about the detected electromagnetic radiation intounmixed image data; and (ii) providing the data through a plurality ofdata channels, in which the unmixed image data in a first data channeldescribes fluorescence signals attributable to the first biomarker andthe unmixed image data in a second data channel describes fluorescencesignals attributable to the second biomarker.

In some embodiments, analyzing the data comprises: (i) dilating, using adilator of the processing circuit, fluorescence signals attributable tothe first biomarker from the first data channel by a predeterminedmargin that is selected to encompass proximally located cells expressingthe second biomarker to generate a dilated first biomarker mask; (ii)determining an interaction area, wherein the interaction area is a firsttotal area for all cells which express the second biomarker and areencompassed within the dilated fluorescence signals attributable to thecells expressing the first biomarker; and (iii) dividing, using aninteraction calculator of the processing circuit, the interaction areaby a normalization factor, and multiplying the resulting quotient by apredetermined factor to arrive at a spatial proximity score.

In some embodiments, the normalization factor is a total area for allcells that have a capacity to express the second biomarker. In someembodiments, the interaction area is determined by combining the dilatedfirst biomarker mask with a mask representative of cells that expressthe second biomarker, determined from signals of the second datachannel. In some embodiments, a third data channel describesfluorescence signals attributable to cell nuclei and a fourth datachannel describes fluorescence signals attributable to tumor area in thesample. In some embodiments, the total area for all cells that have acapacity to express the second biomarker is determined by combining acell mask representative of all cells in the sample, based on signalsfrom the third data channel, and a tumor area mask representative of thetumor area on the sample, based on signals from the fourth data channel.In some embodiments, combining cell mask and the tumor area maskcomprises removing the tumor area mask from the cell mask.

In some embodiments, the processing circuit is further configured tocause the controller to: (i) obtain image data at a low magnificationrepresentative of the concentration of the first or the second biomarkerin the image; (ii) identify areas that include the highest concentrationof the first or the second biomarker; (iii) select a predeterminednumber of the areas including the highest concentration of the first orthe second biomarker; (iv) send instructions to imaging device to obtainhigh magnification image data for the predetermined number of areas;wherein the high magnification image data is provided to the controllerto be analyzed and used to determine the score.

In some embodiments, the low magnification is less than or equal to 10×magnification and wherein the high magnification is greater than 10×. Insome embodiments, the low magnification is 10× magnification and whereinthe high magnification is 40×. In some embodiments, the lowmagnification is 10× magnification and wherein the high magnification is20×. In some embodiments, the low magnification is 4× magnification andwherein the high magnification is 40×. In some embodiments, the lowmagnification is 4× magnification and wherein the high magnification is20×.

In some embodiments, the controller associates the score with metadataassociated with the images of the sample. In some embodiments, thecontroller generates a report including the score. In some embodiments,the controller provides the score to an operator to determineimmunotherapy strategy. In some embodiments, the controller records thescore in a database. In some embodiments, the controller associates thescore with a patient's medical record.

In some embodiments, the electromagnetic radiation source is anincoherent electromagnetic radiation source selected from the groupconsisting of an incandescent lamp, a fluorescent lamp, or a diode. Insome embodiments, the electromagnetic radiation source is a coherentelectromagnetic radiation source. In some embodiments, the systemfurther comprises electromagnetic radiation conditioning opticspositioned to direct electromagnetic radiation from the electromagneticradiation source to the sample. In some embodiments, the electromagneticradiation conditioning optics include an adjustable spectral filterelement configured to provide for illumination of the sample usingdifferent electromagnetic radiation wavelength bands.

In some embodiments, the system further comprises electromagneticradiation collecting optics configured to receive emittedelectromagnetic radiation from the sample and direct the emittedelectromagnetic radiation as output electromagnetic radiation to thedetector. In some embodiments, the electromagnetic radiation collectingoptics include an adjustable spectral filter element configured toselect particular electromagnetic radiation wavelength bands from theelectromagnetic radiation from the sample.

In some embodiments, the detector comprises at least one CCD sensor. Insome embodiments, the detector comprises a photomultiplier tube. In someembodiments, the detector is configured to generate an electrical signalcorresponding to the electromagnetic radiation from the sample andcommunicate the electrical signal to the controller.

In some embodiments, the controller is further configured to sendelectrical signals to one or more of the stage, the electromagneticradiation source, and the detector to adjust at least one property ofthe stage, the electromagnetic radiation source and/or the detector. Insome embodiments, the system further comprises a display device fordisplaying information to the operator. In some embodiments, thedisplayed information is one of parameters of the system, properties ofthe system, and captured images of the sample. In some embodiments, thecontroller displays the score on the display device.

In some embodiments, the information about the detected electromagneticradiation from the imaging device is a plurality of spectral images. Insome embodiments, the plurality of spectral images each correspond to adifferent wavelength of electromagnetic radiation emitted by the sampleand detected by the detector. In some embodiments, each wavelength ofelectromagnetic radiation emitted by the sample corresponds to adifferent fluorophore added to the sample to identify specific featuresin the sample.

In some embodiments, the method provides a superior predictive powercompared to quantitation of expression of the first biomarker orquantitation of expression of the second biomarker. In some embodiments,the predictive power is quantified as a positive predictive value, anegative predictive value, or a combination thereof. In someembodiments, the positive predictive value is 65% or greater. In someembodiments, the positive predictive value is 70% or greater. In someembodiments, the positive predictive value is 75% or greater. In someembodiments, the negative predictive value is 65% or greater. In someembodiments, the negative predictive value is 80% or greater.

In another aspect, disclosed herein are methods of scoring a tissuesample comprising: (i) using an imaging system to obtain image data forthe tissue sample taken from a cancer patient. The imaging systemcomprises a housing comprising a stage for positioning the sample in animaging field, an electromagnetic radiation source for directingelectromagnetic radiation at the sample, and a detector for collectingelectromagnetic radiation output; and a controller comprising memory andan processing circuit having image processing modules. (ii) Analyzing,using the image processing modules, the image data to determine a scorerepresentative of a nearness between a pair of cells, a first member ofthe pair of cells expressing a first biomarker and a second member ofthe pair of cells expressing a second biomarker that is different fromthe first biomarker; and (iii) recording the score in the memory, whichscore when compared to a threshold value is indicative of a likelihoodthat the cancer patient will respond positively to immunotherapy.

In some embodiments, the score is representative of the nearness betweenthe pair of cells is representative of an extent that the pair of cellsare within a predetermined proximity of one another. In someembodiments, analyzing the image date comprises assessing the nearnesson a pixel scale. In some embodiments, the predetermined proximitybetween the pair of cells ranges from about 1 pixel to about 100 pixels.In some embodiments, the predetermined proximity between the pair ofcells ranges from about 5 pixel to about 40 pixels. In some embodiments,the predetermined proximity between the pair of cells ranges from about0.5 μm to about 50 μm. In some embodiments, the predetermined proximitybetween the pair of cells ranges from about 2.5 μm to about 20 μm. Insome embodiments, the score is calculated by obtaining a proximitybetween the boundaries of the pair of cells. In some embodiments, thescore is calculated by obtaining a proximity between the centers of massof the pair of cells. In some embodiments, the score is calculated usingboundary logic based on a perimeter around a selected first cell of thepair of cells. In some embodiments, the score is calculated bydetermining an intersection in the boundaries of the pair of cells. Insome embodiments, the score is calculated by determining an area ofoverlap of the pair of cells.

In some embodiments, generating the image data comprises: (i) separatingthe information about the detected electromagnetic radiation intounmixed image data; and (ii) providing the data through a plurality ofdata channels, in which the unmixed image data in a first data channeldescribes fluorescence signals attributable to the first biomarker andthe unmixed image data in a second data channel describes fluorescencesignals attributable to the second biomarker.

In some embodiments, analyzing the image data comprises: (i) dilating,using a dilator of the processing circuit, fluorescence signalsattributable to the first biomarker from the first data channel by apredetermined margin that is selected to encompass proximally locatedcells expressing the second biomarker to generate a dilated firstbiomarker mask; (ii) determining an interaction area, wherein theinteraction area is a first total area for all cells which express thesecond biomarker and are encompassed within the dilated fluorescencesignals attributable to the cells expressing the first biomarker; and(iii) dividing, using an interaction calculator of the processingcircuit, the interaction area by a normalization factor, and multiplyingthe resulting quotient by a predetermined factor to arrive at a spatialproximity score.

In some embodiments, the normalization factor is a total area for allcells that have a capacity to express the second biomarker. In someembodiments, determining the interaction area comprises combining thedilated first biomarker mask with a mask representative of cells thatexpress the second biomarker, determined from signals of the second datachannel. In some embodiments, a third data channel describesfluorescence signals attributable to cell nuclei and a fourth datachannel describes fluorescence signals attributable to tumor area in thesample.

In some embodiments, the method further comprises determining the totalarea for all cells that have a capacity to express the second biomarkerby combining a cell mask representative of all cells in the sample,based on signals from the third data channel, and a tumor area maskrepresentative of the tumor area on the sample, based on signals fromthe fourth data channel. In some embodiments, combining the cell maskand the tumor area mask comprises removing the tumor area mask from thecell mask.

In some embodiments, the methods further comprise (i) using the imagingsystem to image data at a low magnification representative of theconcentration of the first or the second biomarker in the image; (ii)identifying areas that include the highest concentration of the first orthe second biomarker; (iii) selecting a predetermined number of theareas including the highest concentration of the first or the secondbiomarker; (iv) sending instructions to imaging device to obtain highmagnification image data for the predetermined number of areas; and (iv)wherein the high magnification image data is provided to the controllerto be analyzed and used to determine the score.

In some embodiments, the low magnification is less than or equal to 10×magnification and wherein the high magnification is greater than 10×. Insome embodiments, the low magnification is 4× magnification and whereinthe high magnification is 20×.

In some embodiments, the methods further comprise associating the scorewith metadata associated with the images of the sample. In someembodiments, the methods further comprise generating a report includingthe score. In some embodiments, the methods further comprise providingthe score to a professional to determine immunotherapy strategy. In someembodiments, the methods further comprise recording the score in adatabase. In some embodiments, the methods further comprise associatingthe score with a patient's medical record. In some embodiments, themethods further comprise displaying the score on a display device.

In some embodiments, the method provides a superior predictive powercompared to quantitation of expression of the first biomarker orquantitation of expression of the second biomarker. In some embodiments,the predictive power is quantified as a positive predictive value, anegative predictive value, or a combination thereof. In someembodiments, the positive predictive value is 65% or greater. In someembodiments, the positive predictive value is 70% or greater. In someembodiments, the positive predictive value is 75% or greater. In someembodiments, the negative predictive value is 65% or greater. In someembodiments, the negative predictive value is 80% or greater.

In another aspect, disclosed herein are tissue sample scoring systemscomprising: (i) an imaging device that obtains image data of a tissuesample taken from a cancer patient; and (ii) a controller that receivesimage data from the imaging device and analyzes the data to determine ascore representative of a nearness between a pair of cells, a firstmember of the at least one pair of cells expressing a first biomarkerand a second member of the at least one pair of cells expressing asecond biomarker that is different from the first biomarker; (iii)wherein the score, when compared to a threshold value is indicative of alikelihood that the cancer patient will respond positively toimmunotherapy.

In some embodiments, the imaging device comprises a stage forpositioning the sample in an imaging field, an electromagnetic radiationsource for directing electromagnetic radiation at the sample, and adetector configured to detect electromagnetic radiation from the sample.

In some embodiments, the electromagnetic radiation is selected from thegroup consisting of visible and non-visible light. In some embodiments,the visible light comprises bands of visible light having wavelengthsfalling in the range of about 380 nm to about 720 nm. In someembodiments, the visible light comprises bands of visible light havingwavelengths falling in the range of about 400 nm to about 700 nm. Insome embodiments, the visible light comprises bands of visible lighthaving wavelengths falling in the range of about 380 nm to about 720 nm.

In some embodiments, the emitted or output electromagnetic radiationcomprises one or more bands of visible light from the group consistingof bands including wavelengths falling in the range of about 440 nm toabout 480 nm, about 490 nm to about 550 nm, about 505 nm to about 535nm, about 550 nm to about 595 nm, about 585 nm to about 630 nm, about600 nm to about 640 nm, and about 650 nm to about 710 nm. In someembodiments, the electromagnetic radiation is selected from the groupconsisting of visible and non-visible light. In some embodiments, thevisible light comprises bands of visible light having wavelengthsfalling in the range of about 380 nm to about 720 nm. In someembodiments, the visible light comprises bands of visible light havingwavelengths falling in the range of about 400 nm to about 700 nm. Insome embodiments, the visible light comprises bands of visible lighthaving wavelengths falling in the range of about 380 nm to about 720 nm.

In some embodiments, the method provides a superior predictive powercompared to quantitation of expression of the first biomarker orquantitation of expression of the second biomarker. In some embodiments,the predictive power is quantified as a positive predictive value, anegative predictive value, or a combination thereof. In someembodiments, the positive predictive value is 65% or greater. In someembodiments, the positive predictive value is 70% or greater. In someembodiments, the positive predictive value is 75% or greater. In someembodiments, the negative predictive value is 65% or greater. In someembodiments, the negative predictive value is 80% or greater.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an imaging device for obtaining image dataof a sample.

FIG. 2 is a block diagram of a controller configured to score a samplecomprising tumor tissue taken from a cancer patient, according to anexemplary embodiment.

FIG. 3 is a flowchart of a process for scoring a sample comprising tumortissue, according to an exemplary embodiment.

FIG. 4 is a flowchart of a process for scoring a sample comprising tumortissue, according to a second exemplary embodiment.

FIG. 5 is a flow diagram of the image processing steps used to score asample comprising tumor tissue, according to an exemplary embodiment.

FIG. 6 shows a non-limiting example of an overview of antibodies anddetection reagents used in the preparation of tissue samples for imagingand analysis.

FIG. 7a shows a non-limiting example of all nuclei detected with DAPIwithin an image.

FIG. 7b shows a non-limiting example of a dilated binary mask of allcells within FIG. 7 a.

FIG. 8a shows a non-limiting example of an image of S100 detected with488 dye.

FIG. 8b shows a non-limiting example of a binary mask of all tumor areawithin FIG. 8 a.

FIG. 8c shows a non-limiting example of a mask of all tumor cells withinFIG. 8 a.

FIG. 8d shows a non-limiting example of a mask of all non-tumor cellswithin FIG. 8 a.

FIG. 9a shows a non-limiting example of an image of PD-L1 detected withCy® 5.

FIG. 9b shows a non-limiting example of a binary mask of allPD-L1-positive cells within FIG. 9 a.

FIG. 9c shows a non-limiting example of a mask of all PD-L1-positivetumor cells within FIG. 9 a.

FIG. 9d shows a non-limiting example of a mask of all PD-L1-positivenon-tumor cells within FIG. 9 a.

FIG. 10a shows a non-limiting example of an image of PD-1 detected withCy® 3.5.

FIG. 10b shows a non-limiting example of a binary mask of allPD-1-positive non-tumor cells within FIG. 10 a.

FIG. 11a shows a non-limiting example of an interaction mask of allPD-L1-positive cells and the nearest neighbor cells.

FIG. 11b shows a non-limiting example of an interaction compartment ofthe PD-1-positive cells in close proximity to the PD-L1-positive cells.

FIG. 12a shows a non-limiting example of interaction scores from 26melanoma patients.

FIG. 12b shows a non-limiting example of the maximum interaction scoresfrom the 26 patients of FIG. 12 a.

FIG. 13 shows analysis results based on whole-slide imaging in lieu ofan enrichment algorithm.

FIG. 14 shows a comparison of interaction scores with progression freesurvival of the 26 patients. Note: * indicates uncorrected log-ranktest.

FIG. 15 shows a comparison of PD-L1 expression with progression freesurvival of the patients.

FIG. 16 shows a non-limiting example of a mask of fluorescence signalscorresponding to PD-L1-positive cells (red), PD-1-positive cells(yellow), all tumor cells (green), and all cells (blue) for a positiveresponder to immunotherapy.

FIG. 17 shows a non-limiting example of a mask of fluorescence signalscorresponding to PD-L1-positive cells (red), PD-1-positive cells(yellow), all tumor cells (green), and all cells (blue) for a negativeresponder to immunotherapy.

FIG. 18 shows representative PD-1/PD-L1 interaction scores and values of% biomarker positivity (PBP) for PD-L1 and PD-1 from 38 non-small celllung cancer patients.

FIG. 19 shows a comparison of PD-L1 expression determined using the 22C3FDA-approved IHC assay with progression free survival of the patients.Note: * indicates p-value was determined using uncorrected log-ranktest.

FIG. 20a shows a non-limiting example of interaction scores from 34additional melanoma patients.

FIG. 20b shows a comparison of interaction scores with progression freesurvival of the patients of FIG. 20 a.

FIG. 20c shows the interaction scores from the patients of FIGS. 12a and12b and the patients of FIG. 20 a.

FIG. 20d shows a comparison of interaction scores with progression freesurvival of the patients of FIG. 20c . Note: * indicates the p-value wasdetermined using uncorrected log-rank test.

FIG. 20e shows a comparison of interaction scores with overall survival(OS) of the patients of FIG. 20c . Note: * indicates p-value wascalculated using uncorrected log-rank test.

FIG. 21 shows a non-limiting example of CTLA-4/CD80 interaction scoresfrom 29 metastatic melanoma patients.

FIG. 22 shows a non-limiting example of PD-1/PD-L1 interaction scoresfrom 29 patients with testicular carcinoma.

FIG. 23 shows is a block diagram of a controller configured to derive avalue of biomarker positivity, according to an exemplary embodiment.

FIG. 24 shows is a flowchart of a process for deriving a value ofbiomarker positivity, according to an exemplary embodiment.

FIG. 25 shows is a flowchart of a process for deriving a value ofbiomarker positivity, according to a second exemplary embodiment.

FIG. 26 shows is a flow diagram of the image processing steps used toderive a value of biomarker positivity, according to an exemplaryembodiment.

FIG. 27a shows a non-limiting example of values of % biomarkerpositivity (PBP) for all cells expressing PD-L1 in tissue samples from21 melanoma patients, sorted by increasing PD-L1 expression.

FIG. 27b shows a non-limiting example of values of PBP for all non-tumorcells expressing PD-1 in tissue samples from the same 21 melanomapatients of FIG. 27a , sorted by increasing PD-1 expression.

FIG. 28 shows a comparison of % PD-L1 positivity as ascertained by anautomated cell counting method versus a method described herein.

FIG. 29a shows a non-limiting example of all nuclei detected with DAPIwithin an image.

FIG. 29b shows a non-limiting example of a dilated binary mask of allcells within FIG. 29 a.

FIG. 30a shows a non-limiting example of an image of PD-1 detected withCy® 5.

FIG. 30b shows a non-limiting example of a binary mask of allPD-1-positive cells within FIG. 30 a.

FIG. 31a shows a non-limiting example of an image of CD3 detected withCy® 3.

FIG. 31b shows a non-limiting example of a binary mask of allCD3-positive cells within FIG. 31 a.

FIG. 32 shows a non-limiting example of a binary mask of all cells thatare double positive for PD-1 and CD3.

FIG. 33a shows a non-limiting example of quantitative assessment of CD3+T-cells in tissue samples from DLBCL patients (n=43).

FIG. 33b shows a non-limiting example of quantitative assessment ofCD3+/PD1+ T-cells in tissue samples from DLBCL patients (n=43).

FIG. 34a shows a non-limiting example of quantitative assessment of CD25on NSCLC, gastric, and melanoma tissues.

FIG. 34b shows a non-limiting example of quantitative assessment ofFoxP3 on NSCLC, gastric, and melanoma tissues.

FIG. 35 shows a non-limiting example of quantitative assessment ofCD25+/FoxP3+ T-cells in NSCLC, gastric, and melanoma tissues.

FIG. 36a shows a non-limiting example of quantitative assessment of CD4on NSCLC, gastric, and melanoma tissues.

FIG. 36b shows a non-limiting example of quantitative assessment of CD8on NSCLC, gastric, and melanoma tissues.

FIG. 37a shows a non-limiting example of quantitative assessment ofCD11b+/HLA-DR− phenotype on metastatic melanoma tissues.

FIG. 37b shows a non-limiting example of quantitative assessment ofCD11b+/IDO-1+/HLA-DR− phenotype on metastatic melanoma tissues.

FIG. 38a shows a non-limiting example of quantitative assessment ofIDO-1+/HLA-DR+ phenotype on metastatic melanoma tissues.

FIG. 38b shows a non-limiting example of quantitative assessment ofCD11b+/IDO-1+/HLA-DR+ phenotype on metastatic melanoma tissues.

FIG. 39a shows a non-limiting example of quantitative assessment ofCD11b+/CD33+/ARG1+ phenotype on NSCLC tissues.

FIG. 39b shows a non-limiting example of quantitative assessment ofCD11b+/HLA-DR+/IDO-1+ phenotype on NSCLC tissues.

FIG. 40 shows a non-limiting example of quantitative assessment ofCD11b+/HLA-DR−/IDO-1+ phenotype on NSCLC tissues.

FIG. 41 shows a non-limiting example of quantitative assessment ofCD8+Ki67+ T cells on metastatic melanoma tissues.

FIG. 42a shows a non-limiting example of quantitative assessment ofCD163+ cells on metastatic melanoma tissues.

FIG. 42b shows a non-limiting example of quantitative assessment ofCD68+ cells on metastatic melanoma tissues.

FIG. 42c shows a non-limiting example of quantitative assessment ofCD163+CD68+ cells on metastatic melanoma tissues.

FIG. 43a shows a non-limiting example of quantitative assessment ofLAG-3 positive T cells on DLBCL and NET tissues.

FIG. 43b shows a non-limiting example of quantitative assessment ofTIM-3 positive T cells on DLBCL and NET tissues.

FIG. 44a shows a non-limiting example of quantitative assessment ofCTLA-4 in T cells on melanoma tissues.

FIG. 44b shows a non-limiting example of quantitative assessment of CD80on melanoma tissues.

FIG. 45 shows representative tumor classification based on their immunecontexture.

DETAILED DESCRIPTION

Various embodiments are described hereinafter. It should be noted thatthe specific embodiments are not intended as an exhaustive descriptionor as a limitation to the broader aspects discussed herein. One aspectdescribed in conjunction with a particular embodiment is not necessarilylimited to that embodiment and can be practiced with any otherembodiment(s).

As used herein, “about” will be understood by persons of ordinary skillin the art and will vary to some extent depending upon the context inwhich it is used. If there are uses of the term which are not clear topersons of ordinary skill in the art, given the context in which it isused, “about” will mean up to plus or minus 10% of the particular term.

The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing the elements (especially in the context of thefollowing claims) are to be construed to cover both the singular and theplural, unless otherwise indicated herein or clearly contradicted bycontext. Recitation of ranges of values herein are merely intended toserve as a shorthand method of referring individually to each separatevalue falling within the range, unless otherwise indicated herein, andeach separate value is incorporated into the specification as if it wereindividually recited herein. All methods described herein can beperformed in any suitable order unless otherwise indicated herein orotherwise clearly contradicted by context. The use of any and allexamples, or exemplary language (e.g., “such as”) provided herein, isintended merely to better illuminate the embodiments and does not pose alimitation on the scope of the claims unless otherwise stated. Nolanguage in the specification should be construed as indicating anynon-claimed element as essential.

The term “treating” or “treatment” refers to administering a therapy inan amount, manner, or mode effective to improve a condition, symptom, orparameter associated with a disorder or to prevent progression of adisorder, to either a statistically significant degree or to a degreedetectable to one skilled in the art. An effective amount, manner, ormode can vary depending on the subject and may be tailored to thepatient.

In one aspect, provided herein are systems and methods for performingmethods of scoring a sample comprising tumor tissue taken from a cancerpatient. In another aspect, provided herein are systems and methods forderiving a percent biomarker positivity of a sample of tissue.

In methods disclosed herein, the cancer patient is a mammal. In someembodiments, the mammal is human. In some embodiments, the mammal is nothuman. In further embodiments, the mammal is mouse, rat, guinea pig,dog, cat, or horse.

In methods disclosed herein, tumor tissue is taken from a cancerpatient. The type of cancer includes, but is not limited to, cancers ofthe: circulatory system, for example, heart (sarcoma [angiosarcoma,fibrosarcoma, rhabdomyosarcoma, liposarcoma], myxoma, rhabdomyoma,fibroma, lipoma and teratoma), mediastinum and pleura, and otherintrathoracic organs, vascular tumors and tumor-associated vasculartissue; respiratory tract, for example, nasal cavity and middle ear,accessory sinuses, larynx, trachea, bronchus and lung such as small celllung cancer (SCLC), non-small cell lung cancer (NSCLC), bronchogeniccarcinoma (squamous cell, undifferentiated small cell, undifferentiatedlarge cell, adenocarcinoma), alveolar (bronchiolar) carcinoma, bronchialadenoma, sarcoma, lymphoma, chondromatous hamartoma, mesothelioma;gastrointestinal system, for example, esophagus (squamous cellcarcinoma, adenocarcinoma, leiomyosarcoma, lymphoma), stomach(carcinoma, lymphoma, leiomyosarcoma), gastric, pancreas (ductaladenocarcinoma, insulinoma, glucagonoma, gastrinoma, carcinoid tumors,vipoma), small bowel (adenocarcinoma, lymphoma, carcinoid tumors,Karposi's sarcoma, leiomyoma, hemangioma, lipoma, neurofibroma,fibroma), large bowel (adenocarcinoma, tubular adenoma, villous adenoma,hamartoma, leiomyoma); genitourinary tract, for example, kidney(adenocarcinoma, Wilm's tumor [nephroblastoma], lymphoma, leukemia),bladder and/or urethra (squamous cell carcinoma, transitional cellcarcinoma, adenocarcinoma), prostate (adenocarcinoma, sarcoma), testis(seminoma, teratoma, embryonal carcinoma, teratocarcinoma,choriocarcinoma, sarcoma, interstitial cell carcinoma, fibroma,fibroadenoma, adenomatoid tumors, lipoma); liver, for example, hepatoma(hepatocellular carcinoma), cholangiocarcinoma, hepatoblastoma,angiosarcoma, hepatocellular adenoma, hemangioma, pancreatic endocrinetumors (such as pheochromocytoma, insulinoma, vasoactive intestinalpeptide tumor, islet cell tumor and glucagonoma); bone, for example,osteogenic sarcoma (osteosarcoma), fibrosarcoma, malignant fibroushistiocytoma, chondrosarcoma, Ewing's sarcoma, malignant lymphoma(reticulum cell sarcoma), multiple myeloma, malignant giant cell tumorchordoma, osteochronfroma (osteocartilaginous exostoses), benignchondroma, chondroblastoma, chondromyxofibroma, osteoid osteoma andgiant cell tumors; nervous system, for example, neoplasms of the centralnervous system (CNS), primary CNS lymphoma, skull cancer (osteoma,hemangioma, granuloma, xanthoma, osteitis deformans), meninges(meningioma, meningiosarcoma, gliomatosis), brain cancer (astrocytoma,medulloblastoma, glioma, ependymoma, germinoma [pinealoma], glioblastomamultiform, oligodendroglioma, schwannoma, retinoblastoma, congenitaltumors), spinal cord neurofibroma, meningioma, glioma, sarcoma);reproductive system, for example, gynecological, uterus (endometrialcarcinoma), cervix (cervical carcinoma, pre-tumor cervical dysplasia),ovaries (ovarian carcinoma [serous cystadenocarcinoma, mucinouscystadenocarcinoma, unclassified carcinoma], granulosa-thecal celltumors, Sertoli-Leydig cell tumors, dysgerminoma, malignant teratoma),vulva (squamous cell carcinoma, intraepithelial carcinoma,adenocarcinoma, fibrosarcoma, melanoma), vagina (clear cell carcinoma,squamous cell carcinoma, botryoid sarcoma (embryonal rhabdomyosarcoma),fallopian tubes (carcinoma) and other sites associated with femalegenital organs; placenta, penis, prostate, testis, and other sitesassociated with male genital organs; hematologic system, for example,blood (myeloid leukemia [acute and chronic], acute lymphoblasticleukemia, chronic lymphocytic leukemia, myeloproliferative diseases,multiple myeloma, myelodysplastic syndrome), Hodgkin's disease,non-Hodgkin's lymphoma [malignant lymphoma]; oral cavity, for example,lip, tongue, gum, floor of mouth, palate, and other parts of mouth,parotid gland, and other parts of the salivary glands, tonsil,oropharynx, nasopharynx, pyriform sinus, hypopharynx, and other sites inthe lip, oral cavity and pharynx; skin, for example, malignant melanoma,cutaneous melanoma, basal cell carcinoma, squamous cell carcinoma,Karposi's sarcoma, moles dysplastic nevi, lipoma, angioma,dermatofibroma, and keloids; adrenal glands: neuroblastoma; and othertissues including connective and soft tissue, retroperitoneum andperitoneum, eye, intraocular melanoma, and adnexa, breast, head or/andneck, anal region, thyroid, parathyroid, adrenal gland and otherendocrine glands and related structures, secondary and unspecifiedmalignant neoplasm of lymph nodes, secondary malignant neoplasm ofrespiratory and digestive systems and secondary malignant neoplasm ofother sites, or a combination of one or more thereof.

Examples of immunotherapy include, but are not limited to, monoclonalantibodies (e.g., alemtuzumab or trastuzumab), conjugated monoclonalantibodies (e.g., ibritumomab tiuxetan, brentuximab vendotin, orado-trastuzumab emtansine), bispecific monoclonal antibodies(blinatumomab), immune checkpoint inhibitors (e.g., ipilimumab,pembrolizumab, nivolumab, atezolizumab, or durvalumab), thalidomide,lenalidomide, pomalidomide, and imiquimod, and combinations thereof. Insome embodiments, the immunotherapy comprises immune checkpoint therapy.

Tumors may be classified based on their immune contexture as “hot”(inflamed) or “cold” (non-inflamed) (see FIG. 45). While patientsbearing hot tumors may be expected to respond to certain immunotherapiesand potentially live longer than patients bearing cold tumors, it hasbeen previously unclear to those skilled in the art as to whichbiomarkers correlate with response and survival.

To address this issue, some embodiments of the methods described hereinaid in the identification of cancer patients who respond to one or moreimmunotherapies via expression of immune exhaustion biomarkers (e.g.,PD-1 and PD-L1) and cancer patients who do not respond (i.e.,non-responders) via the presence of cell types known to cause immunesuppression (e.g., CD11b, HLA-DR, IDO-1, ARG1) or highly proliferatingtumor cells devoid of MHC class I expression (e.g., Ki67+, B2M−). Insome embodiments, the methods described herein comprise use of multipleximmunohistochemistry assays (e.g., multiplex FIHC assays) based onspecific immune suppression or activation signatures. Non-limitingexamples of multiplex FIHC assays based on specific immune suppressionor activation signatures are shown in the following table.

Objective Multiplex FIHC Assay T Cell CD3 + PD-1 + PD-L1 SuppressionTumor marker (CK or S100) + PD-1 + PD-L1 CD3 + LAG3 + TIM3 CD3 + CD25 +FOXP3 CTLA-4 + CD80 Myeloid CD11b + HLA-DR(−) IDO-1 (TAM) SuppressionCD11b + HLA-DR (+) ARG1 (fMDSC) T Cell Activation CD3 + CD8 + Ki67 CD3 +CD8 + Granzyme B Immune cell CD3 + CD4 + CD8 Enumeration Tumor marker(CK or S100) + CD16 + CD56 Tumor marker (CK or S100) + CD68 + CD163Identification of Tumor marker (CK or S100) + CD3 (−) B2M (−) Ki67 coldtumors (+)

FIG. 1 is a schematic diagram showing an imaging device 100 foracquiring multiple spectrally resolved images of a sample. Anelectromagnetic radiation (EMR) source 102 provides electromagneticradiation to conditioning optics 104. In some embodiments, theelectromagnetic radiation is visible light. EMR source 102 can be anincoherent light source such as an incandescent lamp, a fluorescentlamp, or a diode. EMR source 102 can also be a coherent source such as alaser source, and the coherent source can provide continuous wave (CW)or pulsed light. EMR source 102 may contain multiple light sourceelements for producing light having a range of wavelengths (e.g.,multiple diodes). The light can be either continuous-wave (CW) ortime-gated (i.e., pulsed) light. Further, light can be provided in aselected portion of the electromagnetic spectrum. For example, light canhave a central wavelength and/or a distribution of wavelengths thatfalls within the ultraviolet, visible, infrared, or other regions of thespectrum. In some embodiments, the light has wavelengths falling in therange of about 380 nm to about 720 nm. EMR source 102 can also includevarious optical elements such as lenses, mirrors, waveplates, andnonlinear crystals, all of which can be used to produce light havingselected characteristics. In general, EMR source 102 includes opticalelements and devices configured to provide light having desiredspectral, spatial, and, in some embodiments, temporal properties.

Conditioning optics 104 can be configured to transform theelectromagnetic radiation, such as visible light, in a number of ways.For example, conditioning optics 104 can spectrally filter light toprovide output light in a selected wavelength region of the spectrum.Alternatively, or in addition, conditioning optics can adjust thespatial distribution of light and the temporal properties of the light.Incident electromagnetic radiation, or incident light, is generated bythe action of the elements of conditioning optics 104 on the EMR.

Incident light is directed to be incident on sample 108 mounted onillumination stage 106. Stage 106 can provide means to secure sample108, such as mounting clips or other fastening devices. Alternatively,stage 106 can include a movable track or belt on which a plurality ofsamples 108 are affixed. A driver mechanism can be configured to movethe track in order to successively translate the plurality of samples,one at a time, through an illumination region on stage 106, whereonincident light impinges on the sample. Stage 106 can further includetranslation axes and mechanisms for translating sample 108 relative to afixed position of illumination stage 106. The translation mechanisms canbe manually operated (e.g., threaded rods) or can be automaticallymovable via electrical actuation (e.g., motorized drivers, piezoelectricactuators).

In response to incident electromagnetic radiation, such as visiblelight, emitted electromagnetic radiation emerges from sample 108.Emitted light can be generated in a number of ways. For example, in someembodiments, emitted light corresponds to a portion of incident lighttransmitted through sample 108. In other embodiments, emitted lightcorresponds to a portion of incident light reflected from sample. In yetfurther embodiments, incident light can be absorbed by sample 108, andthe emitted light corresponds to fluorescence emission from sample 108in response to incident light. In still further embodiments, sample 108can be luminescent, and may produce emitted light even in the absence ofincident light. In some embodiments, emitted light can include lightproduced via two or more of the foregoing mechanisms.

Collecting optics 110 are positioned to received emitted electromagneticradiation, such as emitted light, from sample 108. Collecting optics 110can be configured to collimate emitted light when light is divergent,for example. Collecting optics 110 can also be configured to spectrallyfilter emitted light. Filtering operations can be useful, for example,in order to isolate a portion of emitted light arising via one of themechanisms discussed above from light arising via other processes.Further, collecting optics 110 can be configured to modify the spatialand/or temporal properties of emitted light for particular purposes inembodiments. Light collecting optics 110 transform emitted light intooutput light which is incident on detector 112.

Conditioning optics 104 and collecting optics 110 can include a varietyof optical elements for manipulating the properties of light incidenton, and emitted from, a sample of interest. For example, conditioningoptics 104 and collecting optics 110 can each include spectral filterelements for selecting particular wavelength bands from incident andemitted light. The spectral filter elements can include, for example,interference filters mounted on a filter. In some embodiments,adjustable filter elements based on liquid crystal masks can be used tochange the spectral properties of the incident or emitted light. Liquidcrystal based devices can be controlled by controller 150 viacommunications interface 152.

Conditioning optics 104 and collecting optics 110 can also includeelements such as spatial light masks, spatial light modulators, andoptical pulse shapers in order to manipulate the spatial distribution oflight incident on, or emitted from, a sample. Spatial light modulatorsand other adaptive devices can also be controlled via communicationsinterface 152 by controller 150.

Finally, conditioning optics 104 and collecting optics 110 can includeother common optical elements such as mirrors, lenses, beamsplitters,waveplates, and the like, configured in order to impart selectedcharacteristics to the incident or emitted light.

In general, detector 112 includes one or more measurement devicesconfigured to detect and capture light emitted by a sample as multipleimages of the sample. In embodiments, detector 112 can be configured tomeasure the spatial and/or temporal and/or spectral properties of light.Detector 112 can include devices such as CCD arrays and photomultipliertubes, along with their respective control systems, for acquiring theimages. Detector 112 generates an electrical signal that corresponds tooutput light, and is communicated to controller 150. The adaptiveoptical devices in detector 112 can, in general, be controlled bycontroller 150 via communications interface 152.

Controller 150 includes a communications interface 152 and a processingcircuit 156. In addition to receiving signals corresponding to outputlight detected by detector 112, controller 150 sends electrical signalsto detector 112 to adjust various properties of detector 112, throughcommunications interface 152. For example, if detector 112 includes aCCD sensor, controller 150 can send electrical signals to detector 112to control the exposure time, active area, gain settings, and otherproperties of the CCD sensor.

Controller 150 also communicates with EMR source 102, conditioningoptics 104, stage 106, and collecting optics 110 via communicationsinterface 152. Control system 114 provides electrical signals to each ofthese elements of system 100 to adjust various properties of theelements. For example, electrical signals provided to light source 102can be used to adjust the intensity, wavelength, repetition rate, orother properties of light 122. When the light produced by EMR source 102is pulsed (i.e., time-gated), various properties of the light pulses canbe manipulated according to control signals provided to EMR source 102from controller 150 via communications interface 152. Signals providedto light conditioning optics 104 and light collecting optics 110 caninclude signals for configuring properties of devices that adjust thespatial properties of light (e.g., spatial light modulators) and forconfiguring spectral filtering devices, for example. Signals provided toillumination stage 106 can provide for positioning of sample 108relative to stage 106 and/or for moving samples into position forillumination on stage 106, for example.

Communications interface 152 may include wired or wireless interfaces(e.g., jacks, antennas, transmitters, receivers, transceivers, wireterminals, etc.) for conducting data communications with varioussystems, devices, or networks. For example, communications interface 152may include an Ethernet card and port for sending and receiving data viaan Ethernet-based communications network and/or a WiFi transceiver forcommunicating via a wireless communications network. Communicationsinterface 152 may be configured to communicate via local area networksor wide area networks (e.g., the Internet, a building WAN, etc.) and mayuse a variety of communications protocols (e.g., BACnet, IP, LON, etc.).

Controller 150 may also communicate, via communications interface 152,with a user interface 154. The user interface 154 may be a displaydevice for displaying system properties and parameters, and fordisplaying captured images of sample 108. User interface 154 is providedin order to facilitate operator interaction with, and control over,imaging device 100. Processing circuit 156 includes a storage device,such as memory 160, for storing image data captured using detector 112,and also includes computer software that embodies instructions toprocessor 158 that cause processor 158 to carry out control functions,such as those discussed above and further below, for example. Further,the software instructions cause processor 158 to mathematicallymanipulate the images captured by detector 112. The processing andcalculation of the images are described in greater detail herein,performed either by the processor 116 of the imaging device 100 or anexternal computing system associated with the imaging device 100, suchas controller 200 depicted in FIG. 2 and described below.

In many embodiments, system 100 is configured to acquire multiplespectral images of sample 108. The multiple spectral images maycorrespond to illumination of sample 108 at a variety of selectedwavelengths of light, and detecting an intensity of light eithertransmitted through or reflected by sample 108. Alternatively, themultiple spectral images may correspond to illumination of sample 108with light having similar spectral properties, and collecting multipleimages of sample 108, each image corresponding to a different wavelengthof emitted light. Spectral filtering elements in conditioning optics 104and collecting optics 110 are generally used to obtain the spectrallyresolved data.

In some embodiments, images of sample 108 can be collected in sequence,with adjustments to the configuration of optical components (e.g.,optical filters) between successive captured images. In otherembodiments, multiple images can be captured simultaneously usingdetection systems configured to detect multiple sample views. Forexample, detection systems can be configured to project different viewsof the sample corresponding to different illumination or emissionwavelengths onto a detector such as a CCD camera, and the multiple viewscan be captured simultaneously.

In some embodiments, conditioning optics 104 include an adjustablespectral filter element such as a filter wheel or a liquid crystalspectral filter. The filter element can be configured to provide forillumination of sample 108 using different light wavelength bands. EMRsource 102 can provide light having a broad distribution of spectralwavelength components. A selected region of this broad wavelengthdistribution is allowed to pass as incident light by the filter elementin conditioning optics 104, and directed to be incident on sample 108.An image of light transmitted through sample 108 is recorded by detector112. Subsequently, the wavelength of the filter pass-band inconditioning optics 104 is changed to provide incident light having adifferent wavelength, and an image of light transmitted through sample108 (and corresponding to the new wavelength of incident light) isrecorded. A similar set of spectrally-resolved images can also berecorded by employing an EMR source 102 having multiple source elementsgenerating light of different wavelengths, and alternately turning thedifferent source elements on and off to provide incident light havingdifferent wavelengths.

As discussed previously, the emitted light from sample 108 can alsocorrespond to incident light that is reflected from sample 108. Further,emitted light can correspond to fluorescence emission from sample 108 ifthe sample includes fluorescent chemical structures. For some samples,emitted light can include contributions from multiple sources (i.e.,transmission and fluorescence) and the spectral filtering elements inlight conditioning optics 110 can be used to separate these signalcontributions.

In general, both conditioning optics 104 and collecting optics 110include configurable spectral filter elements. Therefore, spectralresolution can be provided either on the excitation side of sample 108(e.g., via conditioning optics 104) or on the emission side of sample108 (e.g., via collecting optics 110), or both. In any case, the resultof collecting multiple, spectrally resolved images of sample 108 is an“image stack” where each image in the stack is a two-dimensional imageof the sample corresponding to a particular wavelength. Conceptually,the set of images can be visualized as forming a three-dimensionalmatrix, where two of the matrix dimensions are the spatial length andwidth of each of the images, and the third matrix dimension is thespectral wavelength (emission or excitation) to which the imagecorresponds. For this reason, the set of spectrally resolved images canbe referred to as a “spectral cube” of images. As used herein, a “pixel”in such a set of images (or image stack or spectral cube), refers to acommon spatial location for each of the images. Accordingly, a pixel ina set of images includes a value associated with each image at thespatial location corresponding to the pixel.

Other arrangements to obtain spectral images which are known in the artmay be employed, according to the requirements of the sample at hand.

While each spectral image described above typically refers to aparticular wavelength or range of wavelengths (e.g., a spectral band),more generally, each spectral image can correspond to a spectral indexthat may include one or more wavelength bands, or some more complexspectral distribution. For example, such an image can be generated byusing a spectral comb filter. Generally, the image cube will includeseveral spectral images, for example, 10 or more. However, in someembodiments, the image cube may include fewer images, for example, onlytwo or three spectral images. One such example is an red-green-blue(RGB) color image, in which each pixel includes a value associated withthe strength of each of the red, green, and blue colors. Suchinformation may be displayed as a single color image, rather than as aset of separate images; however, the information content is the same asthat in the set of images, and therefore we use the expression “spectralimages” to refer to both cases.

Imaging device 100 can include a wide variety of optical elements anddevices for capturing images and generating image data 130 of a samplethat is used in subsequent sample analysis algorithms, such as methodsand algorithms for scoring a sample comprising tumor tissue taken from acancer patient, described herein. Such imaging devices are described inU.S. Pat. No. 7,555,150 entitled “Classifying Image Features,” which ishereby incorporated by reference in its entirety.

Prior to imaging a sample, the tissue sample may be stained using aplurality of fluorescence tags with affinity for specific biomarkers. Adigital image of the stained sample may be obtained, and the imagefurther analyzed based on the location of the fluorescence tags. Ratherthan whole-image analysis, processing circuit 156 of imaging device 100may include software for causing the controller 150 to perform a fieldof view selection. Therein, fields of view may be prioritized based onthe number of cells that express a first biomarker of interest. Apredetermined number of fields of view may then be further analyzed forfluorescence signals. In some embodiments, the use of four differenttypes of fluorescence tags generates an image of fluorescence signalscorresponding to a first biomarker of interest and an image offluorescence signals corresponding a second biomarker of interest, aswell as to an image of fluorescence signals corresponding to a biomarkerexpressed by all cells and an image of fluorescence signalscorresponding a biomarker expressed by tumor cells.

Examples of fluorophores include, but are not limited to, fluorescein,6-FAM, rhodamine, Texas Red, California Red, iFluor594,tetramethylrhodamine, a carboxyrhodamine, carboxyrhodamine 6F,carboxyrhodol, carboxyrhodamine 110, Cascade Blue, Cascade Yellow,coumarin, Cy2®, Cy3®, Cy3.5®, Cy5®, Cy5.5®, Cy7®, Cy-Chrome, DyLight®350, DyLight® 405, DyLight® 488, DyLight® 549, DyLight® 594, DyLight®633, DyLight® 649, DyLight® 680, DyLight® 750, DyLight® 800,phycoerythrin, PerCP (peridinin chlorophyll-a Protein), PerCP-Cy5.5, JOE(6-carboxy-4′,5′-dichloro-2′,7′-dimethoxyfluorescein), NED, ROX (5-(and-6-)-carboxy-X-rhodamine), HEX, Lucifer Yellow, Marina Blue, OregonGreen 488, Oregon Green 500, Oregon Green 514, Alexa Fluor® 350, AlexFluor® 430, Alexa Fluor® 488, Alexa Fluor® 532, Alexa Fluor® 546, AlexaFluor® 568, Alexa Fluor® 594, Alexa Fluor® 633, Alexa Fluor® 647, AlexaFluor® 660, Alexa Fluor® 680, 7-amino-4-methylcoumarin-3-acetic acid,BODIPY® FL, BODIPY® FL-Br2, BODIPY® 530/550, BODIPY® 558/568, BODIPY®630/650, BODIPY® 650/665, BODIPY® R6G, BODIPY® TMR, BODIPY® TR, OPAL™520, OPAL™ 540, OPAL™ 570, OPAL™ 620, OPAL™ 650, OPAL™ 690, andcombinations thereof. In some embodiments, the fluorophore is selectedfrom the group consisting of DAPI, Cy® 2, Cy® 3, Cy® 3,5, Cy® 5, Cy® 7,FITC, TRITC, a 488 dye, a 555 dye, a 594 dye, Texas Red, and Coumarin.Examples of a 488 dye include, but are not limited to, Alexa Fluor® 488,OPAL™ 520, DyLight® 488, and CF™ 488A. Examples of a 555 dye include,but are not limited to, Alexa Fluor® 555. Examples of a 594 dye include,but are not limited to, Alexa Fluor® 594.

As used herein, a “field of view” refers to a section of a whole-slidedigital image of a tissue sample. In some embodiments, the whole-slideimage has 2-200 predetermined fields of view. In some embodiments, thewhole-slide image has 10-200 predetermined fields of view. In someembodiments, the whole-slide image has 30-200 predetermined fields ofview. In some embodiments, the whole-slide image has 10-150predetermined fields of view. In some embodiments, the whole-slide imagehas 10-100 predetermined fields of view. In some embodiments, thewhole-slide image has 10-50 predetermined fields of view. In someembodiments, the whole-slide image has 10-40 predetermined fields ofview. In some embodiments, the whole-slide image has 10, 15, 20, 25, 30,35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95, or 100, includingincrements therein, predetermined fields of view.

Manipulation of the image data from imaging device 100, for scoring thetissue sample, as described herein, may be carried out by controller200, shown schematically in FIG. 2. Controller 200 is shown to include acommunications interface 202 and a processing circuit 204. Theprocessing circuit 204 is configured to implement steps for scoring thetissue sample. It is contemplated that the elements and functions ofcontroller 200 may be included in controller 150 of imaging device 100,or may be present in a computing system separate from imaging device100.

In some embodiments, the images of fluorescence signals are manipulatedto generate one or more masks of fluorescence signals corresponding tocells within the image. In some embodiments, the one or more masks offluorescence signals comprise one or more selected from the groupconsisting of a mask of all cells within the image, a mask of all cellsthat express the subset biomarker (e.g., all tumor cells), within theimage, a mask of all cells that do not express the subset biomarker(e.g., all non-tumor cells) within the image, a mask of all cellsexpressing a first biomarker of interest within the image, a mask of allcells expressing a second biomarker of interest within the image, and aninteraction mask representing all cells expressing a first biomarker ofinterest within the image as well as proximally located cells expressinga second biomarker of interest. In still further embodiments, theinteraction mask is used to generate an interaction compartment of thecells from all selected fields of view expressing the second biomarkerof interest that were proximally located to the cells expressing thefirst biomarker of interest. The total area of the interactioncompartment may be used to generate a score representative of a spatialproximity between at least one pair of cells, a first member of the atleast one pair of cells expressing the first biomarker and a secondmember of the at least one pair of cells expressing the second biomarkerthat is different from the first biomarker. In some embodiments, thescore indicates the likelihood that the cancer patient will respondpositively to immunotherapy. In some embodiments, the system provides asuperior predictive power compared to a quantitation of expression ofthe first biomarker of interest or a quantitation of expression of thesecond biomarker of interest.

FIG. 3 is a flowchart depicting the steps of one embodiment of a methodfor scoring a sample comprising tumor tissue taken from a cancerpatient. In step 301, image data, such as image data 130, is obtained.Image data may be obtained by an imaging device, such as imaging device100. In step 302, the image data is unmixed such that data specific tovarious types of fluorescence signals are separated into differentchannels. In step 303, data from a first channel is used to generate amask of all cells that are positive for a first biomarker (firstbiomarker mask). The mask of all cells is then dilated (step 304) togenerate a dilated mask representative of a predetermined proximitywithin which an interacting cell (positive for a second biomarker) maybe found. In some embodiments, the first biomarker mask is dilatedbetween 1 and 100 pixels. In step 305, data from a second channel isused to generate a mask of all cells that are positive for the secondbiomarker (second biomarker mask). In step 306, the first biomarker maskand the second biomarker mask are combined to generate an interactionmask identifying cells that are positive for the second biomarker thatare within the predetermined proximity of a cell positive for the firstbiomarker. In step 307, a spatial proximity score is calculated based onthe area of the interaction mask.

FIG. 4 is a second flowchart depicting the steps of a second embodimentof a method for scoring sample comprising tumor tissue taken from acancer patient. In step 401, image data is obtained and in step 402, theimage data is unmixed such that data specific to various types offluorescence signals are separated into different channels. In step 403,data from a first channel is used to generate a mask of all cells in thefield of view and in step 404 data from a second channel is used togenerate a mask of a subset area, such as tumor area in the field ofview. In step 405 the mask of all cells is combined with the subset areamask to generate a mask of subset cells and a mask of non-subset cells.In step 406, data from a third channel is used to generate a mask of allpositive cells that are positive for a first biomarker (first biomarkermask). The mask of all cells is then dilated (step 407) to generate adilated mask representative of a predetermined proximity within which aninteracting cell (i.e., a cell that is positive for a second biomarker)may be found. In some embodiments, the first biomarker mask is dilatedbetween 1 and 100 pixels. In step 408, data from a fourth channel isused to generate a mask of all cells that are positive for the secondbiomarker (second biomarker mask). In step 409, the dilated firstbiomarker mask and the second biomarker mask are combined to generate aninteraction mask identifying cells that are positive for the secondbiomarker and are within the predetermined proximity of a cell positivefor the first biomarker. In step 410, a spatial proximity score iscalculated by dividing the area of the interaction mask by an area ofall subset cells, or of all cells (as indicated by the dotted lines inthe flowchart of FIG. 15 representing use of either input). In someembodiments, the subset cells are cells that are capable of beingpositive for the second biomarker. In some embodiments, the cells thatare capable of being positive for the second biomarker are tumor cellsor non-tumor cells.

In some embodiments, a subset of cells identified by a subset biomarkerand a non-subset of cells corresponds to tumor cells and non-tumorcells, respectively or vice versa. In some embodiments, a subset ofcells identified by a subset biomarker and a non-subset of cellscorresponds to viable cells and non-viable cells, respectively or viceversa. In some embodiments, a subset of cells identified by a subsetbiomarker is a subset of viable cells and a non-subset of cells consistsof the viable cells not included in the subset of viable cells. In someembodiments, a subset of cells identified by a subset biomarker and anon-subset of cells corresponds to T cells and non-T cells, respectivelyor vice versa. In some embodiments, a subset of cells identified by asubset biomarker and a non-subset of cells corresponds to myeloid cellsand non-myeloid cells, respectively or vice versa.

In some embodiment, the spatial proximity score is representative of anearness of a pair of cells. In some embodiments, the nearness of a pairof cells may be determined by a proximity between the boundaries of thepair of cells, a proximity between the centers of mass of the pair ofcells, using boundary logic based on a perimeter around a selected firstcell of the pair of cells, determining an intersection in the boundariesof the pair of cells, and/or by determining an area of overlap of thepair of cells.

In some embodiment, the spatial proximity score is associated withmetadata associated with the images of the sample, included in agenerated report, provided to an operator to determine immunotherapystrategy, recorded in a database, associated with a patient's medicalrecord, and/or displayed on a display device.

In some embodiments, the system provides a superior predictive powercompared to a quantitation of expression of the first biomarker ofinterest or a quantitation of expression of the second biomarker ofinterest.

In the methods disclosed herein, the manipulation of the digital imagesmay be carried out by a computing system comprising a controller, suchas the controller illustrated in the block diagram of FIG. 2, accordingto an exemplary embodiment. Controller 200 is shown to include acommunications interface 202 and a processing circuit 204.Communications interface 202 may include wired or wireless interfaces(e.g., jacks, antennas, transmitters, receivers, transceivers, wireterminals, etc.) for conducting data communications with varioussystems, devices, or networks. For example, communications interface 202may include an Ethernet card and port for sending and receiving data viaan Ethernet-based communications network and/or a WiFi transceiver forcommunicating via a wireless communications network. Communicationsinterface 202 may be configured to communicate via local area networksor wide area networks (e.g., the Internet, a building WAN, etc.) and mayuse a variety of communications protocols (e.g., BACnet, IP, LON, etc.).

Communications interface 202 may be a network interface configured tofacilitate electronic data communications between controller 200 andvarious external systems or devices (e.g., imaging device 102). Forexample, controller 200 may receive imaging data for the selected fieldsof view from the imaging device 102, to analyze the data and calculatethe spatial proximity score (SPS).

Still referring to FIG. 2, processing circuit 204 is shown to include aprocessor 206 and memory 208. Processor 206 may be a general purpose orspecific purpose processor, an application specific integrated circuit(ASIC), one or more field programmable gate arrays (FPGAs), a group ofprocessing components, or other suitable processing components.Processor 506 may be configured to execute computer code or instructionsstored in memory 508 or received from other computer readable media(e.g., CDROM, network storage, a remote server, etc.).

Memory 208 may include one or more devices (e.g., memory units, memorydevices, storage devices, etc.) for storing data and/or computer codefor completing and/or facilitating the various processes described inthe present disclosure. Memory 208 may include random access memory(RAM), read-only memory (ROM), hard drive storage, temporary storage,non-volatile memory, flash memory, optical memory, or any other suitablememory for storing software objects and/or computer instructions. Memory208 may include database components, object code components, scriptcomponents, or any other type of information structure for supportingthe various activities and information structures described in thepresent disclosure. Memory 508 may be communicably connected toprocessor 206 via processing circuit 204 and may include computer codefor executing (e.g., by processor 206) one or more processes describedherein.

Still referring to FIG. 2, controller 200 is shown to receive input froman imaging device 102. The imaging device acquires all of the imagingdata and records it, along with all of the meta-data which describes it.The imaging device will then serialize the data into a stream which canbe read by controller 200. The data stream may accommodate any binarydata stream type such as the file system, a RDBM or direct TCP/IPcommunications. For use of the data stream, controller 200 is shown toinclude a spectral unmixer 210. The spectral unmixer 210 may receiveimage data from an imaging device 102 on which it performs spectralunmixing to unmix an image presenting various wavelengths intoindividual, discrete channels for each band of wavelengths. For example,the image data may be “unmixed” into separate channels for each of thevarious fluorophores used to identify cells or proteins of interest inthe tissue sample. The fluorophore, by way of example only, may be oneor more of the group consisting of DAPI, Cy® 2, Cy® 3, Cy® 3,5, Cy® 5,FITC, TRITC, a 488 dye, a 555 dye, a 594 dye, and Texas Red. In oneexample, one of the channels may include image data that falls within apredetermined band surrounding a wavelength of 461 nm (the maximumemission wavelength for DAPI), to identify nuclei in the image. Otherchannels may include image data for different wavelengths to identifydifferent portions of the tissue sample using different fluorophores.

Controller 200 is also shown to include various maskers, such as cellmasker 212, subset area masker 216, first biomarker masker 22, andsecond biomarker masker 224. These, or other maskers that may beincluded in the controller 200 in other embodiments, are used to receivean unmixed signal from the spectral unmixer 210 and create a mask forthe particular cell or area of interest, dependent on the fluorophoreused to identify certain features of interest in the tissue sample. Tocreate a mask, the maskers (such as cell masker 212, subset area masker216, first biomarker masker 22, and second biomarker masker 224) receiveimage data related to an intensity of each pixel in the field of view.Pixel intensity is directly proportional to the amount of fluorescenceemitted by the sample, which in turn, is directly proportional to theamount of protein biomarker in the sample (when using a fluorophore toidentify a particular biomarker). An absolute threshold may be set basedon the values which exist in the image pixels. All the pixels which aregreater than or equal to the threshold value will be mapped to 1.0, or“on”, and all other pixels will be mapped to 0.0, or “off.” In this way,a binary mask is created to identify the cell or tissue portion ofinterest in the field of view. In other embodiments, a mask is createdusing a lower bound wherein all pixels with an intensity at or above alower bound are accepted and used as the pixel value for the mask. Ifthe intensity is below the lower bound, the pixel value is set to 0.0,or “off”.

In the example flow diagram for masking shown in FIG. 5, it is shownthat the channels for fluorescence signals identifying nuclei and tumorareas (such as DAPI and dye 488 channels, respectively) use the lowerbound protocol (steps 510, 512, 520, 522), while channels foridentifying biomarkers (such as Cy5 and Cy3.5 channels) use a thresholdvalue protocol (steps 530, 540), for providing the mask outputs. Inassociation with the lower bound protocol, there is also a histogramstep to determine the lower bound. In particular, histogram threshold(step 512, 522) produces a threshold of an input image but uses asliding scale to determine the point at which the thresholding occurs.The inputs are the current image and a user defined thresholdpercentage. The latter is used to determine at what percent of the totalintensity the threshold level should be set. Firstly, the intensity ofevery pixel is summed into a total intensity. The threshold percentageis multiplied by this total intensity to obtain a cut-off sum. Finally,all pixels are grouped by intensity (in a histogram) and theirintensities summed from lowest to highest (bin by bin) until the cut-offsum is achieved. The last highest pixel intensity visited in the processis the threshold for the current image. All pixels with intensitiesgreater than that value have their intensities set to maximum while allothers are set to the minimum.

The steps identified as steps 514, 516, 524, 526, 528, 532, 534, 536,542, 544 in FIG. 5 represent intermediary steps that occur in theinitial maskers, such as cell masker 212, subset area masker 216, firstbiomarker masker 222, and second biomarker masker 224. These steps aredefined as follows:

Dilate increases the area of brightest regions in an image. Two inputsare need for dilate. The first is the implicit current image and thesecond is the number of iterations to dilate. It is assumed that onlybinary images are used for the first input. The procedure will operateon continuous images, but the output will not be a valid dilate. Thedilate process begins by first finding the maximum pixel intensity inthe image. Subsequently, each pixel in the image is examined once. Ifthe pixel under investigation has intensity equal to the maximumintensity, that pixel will be drawn in the output image as a circle withiterations radius and centered on the original pixel. All pixels in thatcircle will have intensity equal to the maximum intensity. All otherpixels are copied into the output image without modification.

The fill holes procedure will fill “empty” regions of an image withpixels at maximum intensity. These empty regions are those that have aminimum intensity and whose pixel area (size) is that specified by theuser. The current image and size are the two inputs required. Likedilate this procedure should only be applied to binary images.

Erode processes images in the same fashion as dilate. All functionalityis the same as dilate except that the first step determines the minimumintensity in the image, only pixels matching that lowest intensity arealtered, and the circles used to bloom the found minimum intensitypixels are filled with the lowest intensity value. Like dilate thisprocedure should only be applied to binary images.

Remove Objects. Two inputs are expected: the current image and objectsize. Remove objects is the opposite of the fill holes procedure. Anyregions containing only pixels with maximum intensity filling an arealess than the input object size will be set to minimum intensity andthusly “removed.” This procedure should only be applied to binaryimages; application to continuous images may produce unexpected results.

The output at final steps 518, 529, 538, and 546 are the resultant cellmask, subset area mask (or, in this particular example, the tumor areamask), biomarker 1 cell mask, and biomarker 2 cell mask, respectively.FIG. 5 further depicts the combinations of these resultant masks tocalculate the spatial proximity score. These combinations are describedbelow with reference to the combination maskers of the controller 200,depicted in FIG. 2.

Controller 200 is shown to include combination maskers, such as subsetcell masker 218, non-subset cell masker 220, and interaction masker 230.Subset cell masker performs an And operation, as shown at step 552 inFIG. 5, to combine the output of the cell masker 212 (representative ofall cells in the image) with the output of the subset area masker 216.Accordingly, subset cell masker generates a mask of all subset cells inthe image. In some embodiments, the subset cells are tumor cells. Insome such embodiments, the subset cell masker 218 is a tumor masker andthe non-subset cell masker 220 is a non-tumor masker. This samecombination, using an Out operation performed by non-subset cell masker220 as shown at step 554 in FIG. 5, generates a mask of all non-subsetcells in the sample image. In some embodiments, the non-subset cells arenon-tumor cells.

Before being combined with another mask, the first biomarker mask (fromfirst biomarker masker 222) is dilated by dilator 226. The dilated maskrepresents an area surrounding those cells expressing a first biomarker,so as to identify a space in which cells expressing the second biomarkerwould be within a proper proximity to interact with the cell expressingthe first biomarker. This is represented by steps 556 and 558 of FIG. 5.The flow chart of FIG. 5 shows the dilation taking place in two steps,556 and 558. This may be required when there is a limit to the maximumiterations in each step. For example, there may be a maximum of 10iterations (corresponding to a 10 pixel increase), so when a 20 pixelincrease is needed, the dilation must be split into two subsequentsteps.

Within second biomarker masker 224, the biomarker mask may be combinedwith the non-subset cell mask described above, using an And operation,as shown in step 560 of FIG. 5, to generate a mask of all non-subsetcells that are positive for the first biomarker. This mask is thencombined (step 562) at interaction masker 230 with the dilated mask fromdilator 226 to generate an interaction mask. The interaction maskidentified the non-tumor cells that are positive for the secondbiomarker and that are also within the interaction area, or that overlapthe dilated mask. These identified cells, then, represent the cells thatcould interact with the cells positive for the first biomarker, thusresulting in greater therapy response.

To calculate a spatial proximity score (SPS), the area of theinteraction mask is determined in pixels at the area evaluator 232. Insome embodiments, the area of all the cells that are capable ofexpressing the second biomarker, is determined in pixels at the areaevaluator 234. The cells that are capable of expressing the secondbiomarker may be tumor cells or non-tumor cells. In some embodiments,the area of all cells in the field of view is determined in pixels atthe area evaluator 234. An interaction, or spatial proximity, score isdetermined at the interaction calculator 236 by dividing the area fromarea evaluator 232 by the area from area evaluator 234 and multiplyingby a predetermined factor. As described above, in one embodiment, theequation executed by the interaction calculator 236 is:

${SPS} = {\frac{A_{I}}{A_{C}} \times 10^{4}}$wherein A_(I) is a total interaction area (total area of cellsexpressing the second specific biomarker and encompassed by dilatedfluorescence signals attributable to cells expressing the first specificbiomarker) and A_(C) is the normalization factor. Here, thenormalization is the total area of cells that have a capacity to expressthe second specific biomarker. In some embodiments, the normalizationfactor is the total area of all tumor or non-tumor cells. In someembodiments, the normalization factor is the total area of all cells.

The And procedure is modeled after a binary AND operation, but differsin significant ways. And accepts the current image and a user selectedresultant. The output is an image created by performing a multiplicationof the normalized intensities of matching pixels from the two inputimages. In some applications, image intensity data is alreadynormalized. Therefore, the And procedure is simply a pixel-wisemultiplication of the two images. The two inputs required for Out arethe current image and a user selected resultant. Out removes the secondimage form the first according to the formula A*(1−B/B_(max)) where A isthe current image, B the user selected image to remove, and B_(max) isthe maximum intensity of B. Note that the division of B by B_(max)normalizes B.

In some embodiments, provided herein is an imaging system for scoring asample comprising tumor tissue taken from a cancer patient, the imagingsystem comprising an imaging apparatus comprising a stage forpositioning the sample in an imaging field, an electromagnetic radiationsource for directing electromagnetic radiation at the sample, and adetector configured to detect electromagnetic radiation from the sample,and a controller. The controller comprises a user interface forexchanging information between an operator and the electronic controlsystem and a processing circuit configured to execute instructionsstored on a computer-readable medium. The instructions cause theelectronic control system of the imaging system to: (i) receiveinformation about the detected electromagnetic radiation from theimaging apparatus;

(ii) generate image data based on the detected electromagneticradiation; (iii) analyze the image data to determine a scorerepresentative of a nearness between at least one pair of cells, a firstmember of the least one pair of cells expressing a first biomarker and asecond member of the at least one pair of cells expressing a secondbiomarker that is different from the first biomarker; and (iv) recordthe score, which score when compared to a threshold value is indicativeof a likelihood that the cancer patient will respond positively toimmunotherapy.

In some embodiments, the score representative of a nearness between atleast one pair cells is representative of an extent that the pair ofcells are within a predetermined proximity of one another.

In some embodiments, the first member of the at least one pair of cellscomprises a tumor cell and the second member of the at least one pair ofcells comprises a non-tumor cell. In some embodiments, the non-tumorcell is an immune cell. In some embodiments, the non-tumor cell is astromal cell.

In some embodiments, the first and second members of the at least onepair of cells comprise immune cells.

In some embodiments, the first member of the at least one pair of cellscomprises a tumor cell, a myeloid cell, or a stromal cell and the secondmember of the at least one pair of cells comprises an immune cell. Insome embodiments, the tumor cell, myeloid cell, or stromal cellexpresses PD-L1 and the immune cell expresses PD-1.

In some embodiments, the first member of the at least one pair of cellscomprises a tumor cell and the second member of the at least one pair ofcells comprises an immune cell. In some embodiments, the first member ofthe at least one pair of cells comprises a myeloid cell and the secondmember of the at least one pair of cells comprises an immune cell. Insome embodiments, the first member of the at least one pair of cellscomprises a stromal cell and the second member of the at least one pairof cells comprises an immune cell. In some embodiments, the first memberof the at least one pair of cells expresses PD-L1 and the immune cellexpresses PD-1.

In some embodiments, the first member of the at least one pair of cellsexpresses a first biomarker selected from the group consisting of PD-L1,PD-L2, B7-H3, B7-H4, HLA-DR, Galectin 9, CD80, CD86, 4.1BBL, ICOSL,CD40, OX40L, IDO-1, GITRL, and combinations thereof. In someembodiments, the second member of the at least one pair of cellsexpresses a second biomarker selected from the group consisting of PD-1,TIM3, LAG3, 41BB, OX40, CTLA-4, CD40L, CD28, GITR, ICOS, CD28, andcombinations thereof. In some embodiments, the first member of the atleast one pair of cells expresses a first biomarker selected from thegroup consisting of PD-L1, PD-L2, B7-H3, B7-H4, HLA-DR, Galectin 9,CD80, CD86, 4.1BBL, ICOSL, CD40, OX40L, IDO-1, GITRL, and combinationsthereof, and the second member of the at least one pair of cellsexpresses a second biomarker selected from the group consisting of PD-1,TIM3, LAG3, 41BB, OX40, CTLA-4, CD40L, CD28, GITR, ICOS, CD28, andcombinations thereof.

In some embodiments, the first member of the at least one pair of cellsexpresses PD-L1 and the second member of the at least one pair of cellsexpresses PD-1. In some embodiments, the first member of the at leastone pair of cells expresses PD-L1 and the second member of the at leastone pair of cells expresses CD80. In some embodiments, the first memberof the at least one pair of cells expresses CTLA-4 and the second memberof the at least one pair of cells expresses CD80. In some embodiments,the first member of the at least one pair of cells expresses PD-L2 andthe second member of the at least one pair of cells expresses PD-1. Insome embodiments, the first member of the at least one pair of cellsexpresses CTLA-4 and the second member of the at least one pair of cellsexpresses CD86. In some embodiments, the first member of the at leastone pair of cells expresses LAG-3 and the second member of the at leastone pair of cells expresses HLA-DR. In some embodiments, the firstmember of the at least one pair of cells expresses TIM-3 and the secondmember of the at least one pair of cells expresses Galectin 9. In someembodiments, the first member of the at least one pair of cellsexpresses 41BB and the second member of the at least one pair of cellsexpresses 4.1BBL. In some embodiments, the first member of the at leastone pair of cells expresses OX40 and the second member of the at leastone pair of cells expresses OX40L. In some embodiments, the first memberof the at least one pair of cells expresses CD40 and the second memberof the at least one pair of cells expresses CD40L. In some embodiments,the first member of the at least one pair of cells expresses ICOS andthe second member of the at least one pair of cells expresses ICOSL. Insome embodiments, the first member of the at least one pair of cellsexpresses GITR and the second member of the at least one pair of cellsexpresses GITRL. In some embodiments, the first member of the at leastone pair of cells expresses HLA-DR and the second member of the at leastone pair of cells expresses TCR.

In some embodiments, the first biomarker expressed by the first memberof the at least one pair of cells and the second biomarker expressed bythe second member of the at least one pair of cells interact with oneanother. In some embodiments, the first biomarker expressed by the firstmember of the at least one pair of cells and the second biomarkerexpressed by the second member of the at least one pair of cells do notinteract with one another.

In some embodiments, the spatial proximity between the at least one pairof cells ranges from about 0.5 μm to about 50 μm. In some embodiments,the spatial proximity ranges from 2.5 μm to about 50 μm. In someembodiments, the spatial proximity ranges from 2.5 μm to about 45 μm. Insome embodiments, the spatial proximity ranges from 2.5 μm to about 40μm. In some embodiments, the spatial proximity ranges from 2.5 μm toabout 35 μm. In some embodiments, the spatial proximity ranges from 2.5μm to about 30 μm. In some embodiments, the spatial proximity rangesfrom 2.5 μm to about 25 μm. In some embodiments, the spatial proximityranges from 2.5 μm to about 20 μm. In some embodiments, the spatialproximity ranges from 2.5 μm to about 15 μm. In some embodiments, thespatial proximity ranges from 5 μm to about 50 μm. In some embodiments,the spatial proximity ranges from 5 μm to about 45 μm. In someembodiments, the spatial proximity ranges from 5 μm to about 40 μm. Insome embodiments, the spatial proximity ranges from 5 μm to about 35 μm.In some embodiments, the spatial proximity ranges from 5 μm to about 30μm. In some embodiments, the spatial proximity ranges from 5 μm to about25 μm. In some embodiments, the spatial proximity ranges from 5 μm toabout 20 μm. In some embodiments, the spatial proximity ranges from 5 μmto about 15 μm. In some embodiments, the spatial proximity is about 5,6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24,25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42,43, 44, 45, 46, 47, 48, 49, or 50 μm.

In some embodiments, the spatial proximity between the at least one pairof cells ranges from about 1 pixel to about 100 pixels. In someembodiments, the spatial proximity ranges from about 5 to about 100pixels. In some embodiments, the spatial proximity ranges from about 5to about 90 pixels. In some embodiments, the spatial proximity rangesfrom about 5 to about 80 pixels. In some embodiments, the spatialproximity ranges from about 5 to about 70 pixels. In some embodiments,the spatial proximity ranges from about 5 to about 60 pixels. In someembodiments, the spatial proximity ranges from about 5 to about 50pixels. In some embodiments, the spatial proximity ranges from about 5to about 40 pixels. In some embodiments, the spatial proximity rangesfrom about 5 to about 30 pixels. In some embodiments, the spatialproximity ranges from about 10 to about 100 pixels. In some embodiments,the spatial proximity ranges from about 10 to about 90 pixels. In someembodiments, the spatial proximity ranges from about 10 to about 80pixels. In some embodiments, the spatial proximity ranges from about 10to about 70 pixels. In some embodiments, the spatial proximity rangesfrom about 10 to about 60 pixels. In some embodiments, the spatialproximity ranges from about 10 to about 50 pixels. In some embodiments,the spatial proximity ranges from about 10 to about 40 pixels. In someembodiments, the spatial proximity ranges from about 10 to about 30pixels. In some embodiments, the spatial proximity is about 1, 2, 3, 4,5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23,24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41,42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58 59,60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77,78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95,96, 97, 98, 99, or 100 pixels. In some embodiments, a pixel is 0.5 μmwide.

In some embodiments, generating the image data comprises (i) separatingthe information about the detected electromagnetic radiation intounmixed image data; and (ii) providing the data through a plurality ofdata channels, in which the unmixed image data in a first data channeldescribes fluorescence signals attributable to the first biomarker andthe unmixed image data in a second data channel describes fluorescencesignals attributable to the second biomarker.

In some embodiments, analyzing the data comprises: (i) selecting apredetermined number of fields of view available from the samplecomprising tumor tissue taken from the cancer patient, which is stainedwith a plurality of fluorescence tags, which selection is biased towardselecting fields of view that contain a greater number of cells thatexpress the first biomarker relative to other fields of view; (ii) foreach of the selected fields of view, dilating fluorescence signalsattributable to the first biomarker by a margin sufficient to encompassproximally located cells expressing the second biomarker; and (iii)dividing a first total area for all cells from each of the selectedfields of view, which express the second biomarker and are encompassedwithin the dilated fluorescence signals attributable to the cellsexpressing the first biomarker, with a normalization factor, andmultiplying the resulting quotient by a predetermined factor to arriveat a spatial proximity score.

In some embodiments, analyzing the data comprises: (i) selecting apredetermined number of fields of view available from the samplecomprising tumor tissue taken from the cancer patient, which is stainedwith a plurality of fluorescence tags, which selection is biased towardselecting fields of view that contain a greater number of cells thatexpress the first biomarker relative to other fields of view; (ii) foreach of the selected fields of view, dilating fluorescence signalsattributable to the first biomarker to encompass proximally locatedcells expressing the second biomarker within about 0.5 μm to about 50 μmof a plasma membrane of the cells that express the first biomarker; and(iii) dividing a first total area for all cells from each of theselected fields of view, which express the second biomarker and areencompassed within the dilated fluorescence signals attributable to thecells expressing the first biomarker, with a normalization factor, andmultiplying the resulting quotient by a predetermined factor to arriveat a spatial proximity score.

In some embodiments, analyzing the data comprises: (i) selecting apredetermined number of fields of view available from the samplecomprising tumor tissue taken from the cancer patient, which is stainedwith a plurality of fluorescence tags, which selection is biased towardselecting fields of view that contain a greater number of cells thatexpress the first biomarker relative to other fields of view; (ii) foreach of the selected fields of view, dilating fluorescence signalsattributable to the first biomarker by a margin ranging from about 1 toabout 100 pixels to encompass proximally located cells expressing thesecond biomarker; and (iii) dividing a first total area, as measured inpixels, for all cells from each of the selected fields of view, whichexpress the second biomarker and are encompassed within the dilatedfluorescence signals attributable to the cells expressing the firstbiomarker, with a normalization factor, and multiplying the resultingquotient by a predetermined factor to arrive at a spatial proximityscore.

In some embodiments, analyzing the data comprises: (i) selecting apredetermined number of fields of view available from the samplecomprising tumor tissue taken from the cancer patient, which is stainedwith a plurality of fluorescence tags, which selection is biased towardselecting fields of view that contain a greater number of cells thatexpress the first biomarker relative to other fields of view; (ii) foreach of the selected fields of view, dilating fluorescence signalsattributable to the first biomarker by a margin ranging from about 1 toabout 100 pixels to encompass cells expressing the second biomarkerwithin about 0.5 μm to about 50 μm of a plasma membrane of the cellsthat express the first biomarker; and (iii) dividing a first total area,as measured in pixels, for all cells from each of the selected fields ofview, which express the second biomarker and are encompassed within thedilated fluorescence signals attributable to the cells expressing thefirst biomarker, with a normalization factor, and multiplying theresulting quotient by a predetermined factor to arrive at a spatialproximity score.

In some embodiments, the spatial proximity score is determined by thefollowing equation:

${SPS} = {\frac{A_{I}}{A_{C}} \times 10^{4}}$wherein A_(I) is a total interaction area (total area of cellsexpressing the second specific biomarker and encompassed by dilatedfluorescence signals attributable to cells expressing the first specificbiomarker) and A_(C) is the total area of cells that have a capacity toexpress the second specific biomarker (the normalization factor).

In some embodiments, the spatial proximity score (SPS) is determined bythe following equation:

${SPS} = {\frac{A_{I}}{A_{NT}} \times 10^{4}}$wherein A_(I) is a total interaction area (total area of cellsexpressing the second specific biomarker and encompassed by dilatedfluorescence signals attributable to cells expressing the first specificbiomarker) and A_(NT) is the total area of non-tumor cells.

In some embodiments, the spatial proximity score is determined by thefollowing equation:

${SPS} = {\frac{A_{I}}{A_{T}} \times 10^{4}}$wherein A_(I) is a total interaction area (total area of cellsexpressing the second specific biomarker and encompassed by dilatedfluorescence signals attributable to cells expressing the first specificbiomarker) and A_(T) is the total area of all cells.

In some embodiments, four fluorescence tags, each specific to adifferent biomarker, are used in the determining step. In furtherembodiments, a first fluorescence tag is associated with the firstbiomarker, a second fluorescence tag is associated with the secondbiomarker, a third fluorescence tag is associated with a thirdbiomarker, and a fourth fluorescence tag is associated with a fourthbiomarker. In some embodiments, the first biomarker comprises a tumorand non-tumor marker. In some embodiments, the second biomarkercomprises a non-tumor marker. In some embodiments, the first biomarkercomprises a tumor and non-tumor marker, and the second biomarkercomprises a non-tumor marker. In some embodiments, the third biomarkeris expressed by all cells. In some embodiments, the fourth biomarker isexpressed only in tumor cells. In some embodiments, the third biomarkeris expressed by all cells and the fourth biomarker is expressed only intumor cells. In some embodiments, one or more fluorescence tags comprisea fluorophore conjugated to an antibody having a binding affinity for aspecific biomarker or another antibody. In some embodiments, one or morefluorescence tags are fluorophores with affinity for a specificbiomarker.

In some embodiments, the fluorescence signals attributable to the firstbiomarker are dilated by a margin ranging from about 1 to about 100pixels. In some embodiments, the margin is from about 5 to about 100pixels. In some embodiments, the margin is from about 5 to about 90pixels. In some embodiments, the margin is from about 5 to about 80pixels. In some embodiments, the margin is from about 5 to about 70pixels. In some embodiments, the margin is from about 5 to about 60pixels. In some embodiments, the margin is from about 5 to about 50pixels. In some embodiments, the margin is from about 5 to about 40pixels. In some embodiments, the margin is from about 5 to about 30pixels. In some embodiments, the margin is from about 10 to about 100pixels. In some embodiments, the margin is from about 10 to about 90pixels. In some embodiments, the margin is from about 10 to about 80pixels. In some embodiments, the margin is from about 10 to about 70pixels. In some embodiments, the margin is from about 10 to about 60pixels. In some embodiments, the margin is from about 10 to about 50pixels. In some embodiments, the margin is from about 10 to about 40pixels. In some embodiments, the margin is from about 10 to about 30pixels. In some embodiments, the margin is about 1, 2, 3, 4, 5, 6, 7, 8,9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26,27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44,45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58 59, 60, 61, 62,63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80,81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98,99, or 100 pixels. In some embodiments, a pixel is 0.5 μm wide.

In some embodiments, dilating fluorescence signals attributable to thefirst biomarker encompasses proximally located cells expressing thesecond biomarker within about 0.5 μm to about 50 μm of a plasma membraneof the cells that express the first biomarker. In some embodiments,dilating fluorescence signals attributable to the first biomarkerencompasses proximally located cells expressing the second biomarkerwithin about 2.5 μm to about 50 μm of a plasma membrane of the cellsthat express the first biomarker. In some embodiments, dilatingfluorescence signals attributable to the first biomarker encompassesproximally located cells expressing the second biomarker within about2.5 μm to about 45 μm of a plasma membrane of the cells that express thefirst biomarker. In some embodiments, dilating fluorescence signalsattributable to the first biomarker encompasses proximally located cellsexpressing the second biomarker within about 2.5 μm to about 40 μm of aplasma membrane of the cells that express the first biomarker. In someembodiments, dilating fluorescence signals attributable to the firstbiomarker encompasses proximally located cells expressing the secondbiomarker within about 2.5 μm to about 35 μm of a plasma membrane of thecells that express the first biomarker. In some embodiments, dilatingfluorescence signals attributable to the first biomarker encompassesproximally located cells expressing the second biomarker within about2.5 μm to about 30 μm of a plasma membrane of the cells that express thefirst biomarker. In some embodiments, dilating fluorescence signalsattributable to the first biomarker encompasses proximally located cellsexpressing the second biomarker within about 2.5 μm to about 25 μm of aplasma membrane of the cells that express the first biomarker. In someembodiments, dilating fluorescence signals attributable to the firstbiomarker encompasses proximally located cells expressing the secondbiomarker within about 2.5 μm to about 20 μm of a plasma membrane of thecells that express the first biomarker. In some embodiments, dilatingfluorescence signals attributable to the first biomarker encompassesproximally located cells expressing the second biomarker within about2.5 μm to about 15 μm of a plasma membrane of the cells that express thefirst biomarker. In some embodiments, dilating fluorescence signalsattributable to the first biomarker encompasses proximally located cellsexpressing the second biomarker within about 5 μm to about 50 μm of aplasma membrane of the cells that express the first biomarker. In someembodiments, dilating fluorescence signals attributable to the firstbiomarker encompasses proximally located cells expressing the secondbiomarker within about 5 μm to about 45 μm of a plasma membrane of thecells that express the first biomarker. In some embodiments, dilatingfluorescence signals attributable to the first biomarker encompassesproximally located cells expressing the second biomarker within about 5μm to about 40 μm of a plasma membrane of the cells that express thefirst biomarker. In some embodiments, dilating fluorescence signalsattributable to the first biomarker encompasses proximally located cellsexpressing the second biomarker within about 5 μm to about 35 μm of aplasma membrane of the cells that express the first biomarker. In someembodiments, dilating fluorescence signals attributable to the firstbiomarker encompasses proximally located cells expressing the secondbiomarker within about 5 μm to about 30 μm of a plasma membrane of thecells that express the first biomarker. In some embodiments, dilatingfluorescence signals attributable to the first biomarker encompassesproximally located cells expressing the second biomarker within about 5μm to about 25 μm of a plasma membrane of the cells that express thefirst biomarker. In some embodiments, dilating fluorescence signalsattributable to the first biomarker encompasses proximally located cellsexpressing the second biomarker within about 5 μm to about 20 μm of aplasma membrane of the cells that express the first biomarker. In someembodiments, dilating fluorescence signals attributable to the firstbiomarker encompasses proximally located cells expressing the secondbiomarker within about 5 μm to about 15 μm of a plasma membrane of thecells that express the first biomarker. In some embodiments, dilatingfluorescence signals attributable to the first biomarker encompassesproximally located cells expressing the second biomarker within about 5,6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24,25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42,43, 44, 45, 46, 47, 48, 49, or 50 μm of a plasma membrane of the cellsthat express the first biomarker. In some embodiments, the secondbiomarker on the proximally located cells is in direct contact with thefirst biomarker.

In some embodiments, the first total area for all cells from each of theselected fields of view, which express the second biomarker, is measuredin pixels.

In some embodiments, the normalization factor is a second total area forall non-tumor cells from each of the selected fields of view. In someembodiments, the second total area is measured in pixels. In someembodiments, both the first total area and the second total areameasured in pixels.

In some embodiments, the normalization factor is a second total area forall cells from each of the selected fields of view which have thecapacity to express the second biomarker. In some embodiments, thesecond total area is measured in pixels. In some embodiments, both thefirst total area and the second total area measured in pixels.

In some embodiments, the normalization factor is a second total area forall cells from each of the selected fields of view. In some embodiments,the second total area is measured in pixels. In some embodiments, boththe first total area and the second total area measured in pixels.

In some embodiments, the threshold score is about 500 to about 5000. Insome embodiments, the threshold score is about 500 to about 4500. Insome embodiments, the threshold score is about 500 to about 4000. Insome embodiments, the threshold score is about 500 to about 3500. Insome embodiments, the threshold score is about 500 to about 3000. Insome embodiments, the threshold score is about 500 to about 2500. Insome embodiments, the threshold score is about 500 to about 2000. Insome embodiments, the threshold score is about 500 to about 1500. Insome embodiments, the threshold score is about 500 to about 1000. Insome embodiments, the threshold score is about 500, 550, 600, 650, 700,750, 800, 850, 900, 950, 1000, 1100, 1200, 1300, 1400, 1500, 1600, 1700,1800, 1900, 2000, 2100, 2200, 2300, 2400, 2500, 2600, 2700, 2800, 2900,3000, 3100, 3200, 3300, 3400, 3500, 3600, 3700, 3800, 3900, 4000, 4100,4200, 4300, 4400, 4500, 4600, 4700, 4800, 4900, or 5000, includingincrements therein. In some embodiments, the threshold score is about500, 550, 600, 650, 700, 750, 800, 850, 900, 950, 1000, 1100, 1200,1300, 1400, 1500, 1600, 1700, 1800, 1900, 2000, 2100, 2200, 2300, 2400,2500, 2600, 2700, 2800, 2900, 3000, 3100, 3200, 3300, 3400, 3500, 3600,3700, 3800, 3900, 4000, 4100, 4200, 4300, 4400, 4500, 4600, 4700, 4800,4900, or 5000, including increments therein, plus or minus 100.

In some embodiments, the predetermined factor is from about 10 to about10⁵. In some embodiments, the predetermined factor is from about 10² toabout 10⁵. In some embodiments, the predetermined factor is from about10³ to about 10⁵. In some embodiments, the predetermined factor is fromabout 10⁴ to about 10⁵. In some embodiments, the predetermined factor isabout 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 200, 300, 400, 500, 600,700, 800, 900, 1000, 1500, 2000, 2500, 3000, 3500, 4000, 4500, 5000,5500, 6000, 6500, 7000, 7500, 8000, 8500, 9000, 9500, 10000, 20000,30000, 40000, 50000, 60000, 70000, 80000, 90000, or 10⁵, includingincrements therein.

In some embodiments, the predictive power is quantified as a positivepredictive value, a negative predictive value, or a combination thereof.A positive predictive value is calculated by dividing the number ofpatients who respond to treatment with scores above the threshold scoreby the total number of patients who respond to treatment. A negativepredictive value is calculated by dividing the number of patients who donot respond to treatment with scores below the threshold score by thetotal number of patients who do not respond to treatment.

In some embodiments, the positive predictive value is greater than 60%.In some embodiments, the positive predictive value is 65% or greater. Insome embodiments, the positive predictive value is 70% or greater. Insome embodiments, the positive predictive value is 75% or greater. Insome embodiments, the positive predictive value is 80% or greater. Insome embodiments, the positive predictive value is about 50, 51, 52, 53,54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71,72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89,90, 91, 92, 93, 94, 95, 96, 97, 98, or 99%, including incrementstherein.

In some embodiments, the negative predictive value is 60% or greater. Insome embodiments, the negative predictive value is 65% or greater. Insome embodiments, the negative predictive value is 70% or greater. Insome embodiments, the negative predictive value is 75% or greater. Insome embodiments, the negative predictive value is 80% or greater. Insome embodiments, the negative predictive value is about 50, 51, 52, 53,54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71,72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89,90, 91, 92, 93, 94, 95, 96, 97, 98, or 99%, including incrementstherein.

In another aspect, disclosed herein are methods utilizing a systemcomprising an imaging device and a controller for determining a scorerepresentative of a spatial proximity between at least one pair of cellsselected from among a plurality of cells present in a predeterminednumber of fields of view available from a sample comprising tumortissue, which sample is taken from a cancer patient, the methodcomprising: (i) selecting a predetermined number of fields of viewavailable from the sample comprising tumor tissue taken from the cancerpatient, which is stained with a plurality of fluorescence tags, whichselection is biased toward selecting fields of view that contain agreater number of cells that express a first specific biomarker relativeto other fields of view; (ii) for each of the selected fields of view,dilating fluorescence signals attributable to the first specificbiomarker to encompass proximally located cells expressing a secondspecific biomarker; and (iii) dividing a first total area for all cellsfrom each of the selected fields of view, which express the secondspecific biomarker and are encompassed within the dilated fluorescencesignals attributable to the cells expressing the first specificbiomarker, with a normalization score, and multiplying the resultingquotient by a predetermined factor to arrive at a spatial proximityscore. In some embodiments, the method provides a superior predictivepower compared to a quantitation of expression of the first specificbiomarker or a quantitation of expression of the second specificbiomarker.

In another aspect, disclosed herein methods utilizing a systemcomprising an imaging device and a controller for determining a scorerepresentative of a spatial proximity between at least one pair of cellsselected from among a plurality of cells present in a predeterminednumber of fields of view available from a sample comprising tumortissue, which sample is taken from a cancer patient, the methodcomprising: (i) selecting a predetermined number of fields of viewavailable from the sample comprising tumor tissue taken from the cancerpatient, which is stained with a plurality of fluorescence tags, whichselection is biased toward selecting fields of view that contain agreater number of cells that express a first biomarker relative to otherfields of view; (ii) for each of the selected fields of view, dilatingfluorescence signals attributable to the first biomarker to encompasscells expressing a second biomarker within about 0.5 μm to about 50 μmof a plasma membrane of the cells that express the first biomarker; and(iii) dividing a first total area for all cells from each of theselected fields of view, which express the second biomarker and areencompassed within the dilated fluorescence signals attributable to thecells expressing the first biomarker, with a normalization factor, andmultiplying the resulting quotient by a predetermined factor to arriveat a spatial proximity score. In some embodiments, the method provides asuperior predictive power compared to a quantitation of expression ofthe first specific biomarker or a quantitation of expression of thesecond specific biomarker.

In another aspect, disclosed herein are methods utilizing a systemcomprising an imaging device and a controller for determining a scorerepresentative of a spatial proximity between at least one pair of cellsselected from among a plurality of cells present in a predeterminednumber of fields of view available from a sample comprising tumortissue, which sample is taken from a cancer patient, the methodcomprising: (i) selecting a predetermined number of fields of viewavailable from the sample comprising tumor tissue taken from the cancerpatient, which is stained with a plurality of fluorescence tags, whichselection is biased toward selecting fields of view that contain agreater number of cells that express a first biomarker relative to otherfields of view; (ii) for each of the selected fields of view, dilatingfluorescence signals attributable to the first biomarker by a marginranging from about 1 to about 100 pixels to encompass proximally locatedcells expressing a second biomarker; and (iii) dividing a first totalarea, as measured in pixels, for all cells from each of the selectedfields of view, which express the second biomarker and are encompassedwithin the dilated fluorescence signals attributable to the cellsexpressing the first biomarker, with a normalization factor, andmultiplying the resulting quotient by a predetermined factor to arriveat a spatial proximity score. In some embodiments, the method provides asuperior predictive power compared to a quantitation of expression ofthe first specific biomarker or a quantitation of expression of thesecond specific biomarker.

In another aspect, disclosed herein are methods utilizing a systemcomprising an imaging device and a controller for determining a scorerepresentative of a spatial proximity between at least one pair of cellsselected from among a plurality of cells present in a predeterminednumber of fields of view available from a sample comprising tumortissue, which sample is taken from a cancer patient, the methodcomprising: (i) selecting a predetermined number of fields of viewavailable from the sample comprising tumor tissue taken from the cancerpatient, which is stained with a plurality of fluorescence tags, whichselection is biased toward selecting fields of view that contain agreater number of cells that express a first biomarker relative to otherfields of view; (ii) for each of the selected fields of view, dilatingfluorescence signals attributable to the first biomarker by a marginranging from about 1 to about 100 pixels to encompass cells expressing asecond biomarker within about 0.5 μm to about 50 μm of a plasma membraneof the cells that express the first biomarker; and (iii) dividing afirst total area, as measured in pixels, for all cells from each of theselected fields of view, which express the second biomarker and areencompassed within the dilated fluorescence signals attributable to thecells expressing the first biomarker, with a normalization factor, andmultiplying the resulting quotient by a predetermined factor to arriveat a spatial proximity score. In some embodiments, the method provides asuperior predictive power compared to a quantitation of expression ofthe first specific biomarker or a quantitation of expression of thesecond specific biomarker.

In some embodiments, the spatial proximity score (SPS) is determined bythe following equation:

${SPS} = {\frac{A_{I}}{A_{NT}} \times 10^{4}}$wherein A_(I) is a total interaction area (total area of cellsexpressing the second specific biomarker and encompassed by dilatedfluorescence signals attributable to cells expressing the first specificbiomarker) and A_(NT) is the total area of non-tumor cells.

In some embodiments, the spatial proximity score is determined by thefollowing equation:

${SPS} = {\frac{A_{I}}{A_{C}} \times 10^{4}}$wherein A_(I) is a total interaction area (total area of cellsexpressing the second specific biomarker and encompassed by dilatedfluorescence signals attributable to cells expressing the first specificbiomarker) and A_(C) is the total area of cells that have a capacity toexpress the second specific biomarker.

In another aspect, disclosed are methods utilizing a system comprisingan imaging device and a controller for scoring a sample comprising tumortissue from a cancer patient that are used in methods of treating cancerin the patient. In some embodiments, the methods of scoring a samplecomprising tumor tissue from a cancer patient are performed prior toadministration of immunotherapy. In some embodiments, the methodprovides a superior predictive power compared to a quantitation ofexpression of the first specific biomarker or a quantitation ofexpression of the second specific biomarker.

In some embodiments, the spatial proximity score is determined by thefollowing equation:

${SPS} = {\frac{A_{I}}{A_{T}} \times 10^{4}}$wherein A_(I) is a total interaction area (total area of cellsexpressing the second specific biomarker and encompassed by dilatedfluorescence signals attributable to cells expressing the first specificbiomarker) and A_(T) is the total area of all cells.

In some embodiments, disclosed herein are methods utilizing a systemcomprising an imaging device and a controller for treating cancer in apatient in need thereof, the method comprising: (a) scoring a samplecomprising tumor tissue taken from the patient comprising (i) using thesample comprising tumor tissue taken from the patient, determining ascore representative of a spatial proximity between at least one pair ofcells, a first member of the at least one pair of cells expressing afirst biomarker and a second member of the at least one pair of cellsexpressing a second biomarker that is different from the firstbiomarker; and (ii) recording the score; (b) comparing the score to athreshold value; and (b) administering immunotherapy to the patient ifthe score when compared to the threshold value is indicative of alikelihood that the patient will respond positively to theimmunotherapy. In some embodiments, the determining step is as describedherein. In some embodiments, the method provides a superior predictivepower compared to a quantitation of expression of the first specificbiomarker or a quantitation of expression of the second specificbiomarker.

In some embodiments, disclosed herein are methods of scoring a tissuesample comprising: (i) using an imaging system to obtain image data forthe tissue sample taken from a cancer patient, the imaging systemcomprising: a housing comprising a stage for positioning the sample inan imaging field, an electromagnetic radiation source for directingelectromagnetic radiation at the sample, and a detector for collectingelectromagnetic radiation output; and an electronic control systemcomprising memory and an processing circuit having image processingmodules; (ii) analyzing, using the image processing modules, the imagedata to determine a score representative of a nearness between a pair ofcells, a first member of the pair of cells expressing a first biomarkerand a second member of the pair of cells expressing a second biomarkerthat is different from the first biomarker; and (iii) recording thescore in the memory, which score when compared to a threshold value isindicative of a likelihood that the cancer patient will respondpositively to immunotherapy. In some embodiments, the method provides asuperior predictive power compared to a quantitation of expression ofthe first specific biomarker or a quantitation of expression of thesecond specific biomarker.

In some embodiments, disclosed herein are tissue sample scoring systemscomprising: an imaging device that obtains image data of a tissue sampletaken from a cancer patient; and a controller that receives image datafrom the imaging device and analyzes the data to determine a scorerepresentative of a nearness between a pair of cells, a first member ofthe at least one pair of cells expressing a first biomarker and a secondmember of the at least one pair of cells expressing a second biomarkerthat is different from the first biomarker, wherein the score, whencompared to a threshold value is indicative of a likelihood that thecancer patient will respond positively to immunotherapy. In someembodiments, the method provides a superior predictive power compared toa quantitation of expression of the first specific biomarker or aquantitation of expression of the second specific biomarker.

EXAMPLES Example 1. Sample Preparation, Imaging, and Analysis of Imagingfor Melanoma Tissue Samples from Human Patients

Sample Preparation.

Formalin fixed paraffin embedded (FFPE) tissue samples were dewaxed. Theslides were then rehydrated through a series of xylene to alcohol washesbefore incubating in distilled water. Heat-induced antigen retrieval wasthen performed using elevated pressure and temperature conditions,allowed to cool, and transferred to Tris-buffered saline. Staining wasthen performed where the following steps were carried out. First,endogenous peroxidase was blocked followed by incubation with aprotein-blocking solution to reduce nonspecific antibody staining. Next,the slides were stained with a mouse anti-PD1 primary antibody. Slideswere then washed before incubation with an anti-mouse HRP secondaryantibody. Slides were washed and then PD-1 staining was detected usingTSA+Cy® 3.5 (Perkin Elmer). Any residual HRP was then quenched using twowashes of fresh 100 mM benzhydrazide with 50 mM hydrogen peroxide. Theslides were again washed before staining with a rabbit anti-PD-L1primary antibody. Slides were washed and then incubated with a cocktailof anti-rabbit HRP secondary antibody plus mouse anti-S100 directlylabeled with 488 dye and 4′,6-diamidino-2-phenylindole (DAPI). Slideswere washed and then PD-L1 staining was detected using TSA-Cy® 5 (PerkinElmer). Slides were washed a final time before they were cover-slippedwith mounting media and allowed to dry overnight at room temperature. Aschematic overview of the antibodies and detection reagents is shown inFIG. 6. Alternatively, slides were stained with anti-CD8 primaryantibody in place of anti-PD1 primary antibody.

Sample Imaging and Analysis.

Fluorescence images were then acquired using the Vectra 2 IntelligentSlide Analysis System using the Vectra software version 2.0.8 (PerkinElmer). First, monochrome imaging of the slide at 4× magnification usingDAPI was conducted. An automated algorithm (developed using inForm) wasused to identify areas of the slide containing tissue.

The areas of the slide identified as containing tissue were imaged at 4×magnification for channels associated with DAPI (blue), FITC (green),and Cy® 5 (red) to create RGB images. These 4× images were processedusing an automated enrichment algorithm (developed using inForm) infield of view selector 101 to identify and rank possible 20×magnification fields of view according to the highest Cy® 5 expression.

The top 40 fields of view were imaged at 20× magnification across DAPI,FITC, Texas Red, and Cy® 5 wavelengths. Raw images were reviewed foracceptability, and images that were out of focus, lacked any tumorcells, were highly necrotic, or contained high levels of fluorescencesignal not associated with expected antibody localization (i.e.,background staining) were rejected prior to analysis. Accepted imageswere processed using AQUAduct (Perkin Elmer), wherein each fluorophorewas spectrally unmixed by spectral unmixer 210 into individual channelsand saved as a separate file.

The processed files were further analyzed using AQUAnalysis™ or througha fully automated process using AQUAserve™. Details were as follows.

Each DAPI image was processed by cell masker 212 to identify all cellnuclei within that image (FIG. 7a ), and then dilated by 3 pixels torepresent the approximate size of an entire cell. This resulting maskrepresented all cells within that image (FIG. 7b ).

S100 (tumor cell marker for melanoma) detected with 488 dye (FIG. 8a )was processed by subset masker 216 to create a binary mask of all tumorarea within that image (FIG. 8b ). Overlap between this mask and themask of all cells created a new mask for tumor cells (FIG. 8c ), usingtumor cell masker 218.

Similarly, absence of the tumor cell marker in combination with the maskof all nuclei created a new mask for all non-tumor cells (FIG. 8d ),performed using non-tumor cell masker 220.

Each Cy® 5 image (FIG. 9a ) was processed by first biomarker masker 222and overlapped with the mask of all cells to create a binary mask of allcells that are PD-L1-positive (FIG. 9b ). Overlapping the biomarker maskwith the mask of all cells eliminated noise pixels that may be falselyidentified in the mask as biomarker positive cells.

Each Cy® 3.5 image (FIG. 10a ) was processed by second biomarker masker224 to create a binary mask for PD-1-positive cells and overlapped withthe mask of all non-tumor cells to create a binary mask of all non-tumorcells that are PD-1-positive (FIG. 10b ). Overlapping the biomarker maskwith the mask of all non-tumor cells eliminated noise pixels that may befalsely identified in the mask as biomarker positive cells.

The binary mask of all PD-L1-positive cells was dilated using seconddilator 226 to create an interaction mask encompassing the nearestneighbor cells (e.g., cells with PD-1) (FIG. 11a ). This interactionmask was combined with a binary mask of all PD-1-positive non-tumorcells using interaction masker 230 to create an interaction compartmentof the PD-1-positive cells in close enough proximity to thePD-L1-positive cells such that PD-1 is likely interacting with PD-L1(FIG. 11b ).

The total area from all accepted fields (up to 40 fields of view) forthe interaction compartment and the total area of the non-tumor cellswas calculated in area evaluators 232, 234 respectively. The total areafrom all accepted fields of view for the interaction compartment wasdivided by the total area of the non-tumor cells and multiplied by afactor of 10,000, using the interaction calculator 236 to create a wholenumber representing an interaction score for each specimen. PD-L1 andPD-1 measurements were highly reproducible (R²=0.98 and 0.97,respectively). A broad range of PD-L1 and PD-1 expression andinteraction scores were observed in archival clinical specimens (n=53).In a cohort of 26 advanced melanoma patients treated with nivolumab(n=5) or pembrolizumab (n=21), the PD-1/PD-L1 interaction score wasfound to reliably distinguish responders from non-responders (p=0.01)while PD-L1 alone (p=0.07) or CD8 alone (p=0.23) did not. Additionally,patients exhibiting higher PD-1/PD-L1 interaction scores had superiorresponse rates (82% vs. 20%, p=0.01). Patients with high PD-1/PD-L1interaction scores experienced longer median progression-free survival(p=0.059), and fewer deaths (22% vs 58%) compared with patients havinglower PD-1/PD-L1 interaction scores. These results suggest that thismethod of scoring the tissue sample to obtain PD-1/PD-L1 interactionscores provides a superior predictive power (82% Positive PredictiveValue, 80% Negative Predictive Value) compared with PD-L1 expressionalone.

Representative scores from the 26 patients are shown in FIG. 12a . Basedon the data, a threshold of 800-900 was selected to indicate likelihoodof response to treatment.

Alternatively, the interaction score was calculated for each individualfield of view and the maximum score for each patient is shown in FIG.12b . Based on the maximum score, a threshold of 1900 was selected toindicate likelihood of response to treatment.

To assess the effect of the enrichment algorithm on the interactionscore, the above-mentioned procedures were performed using whole-slideimaging in lieu of the enrichment algorithm (see FIG. 13). Whenwhole-slide image analysis was performed, there was no longer astatistically significant difference between the patients who respondedto anti-PD1 therapy and those who did not. As such, a threshold couldnot be determined with this analysis.

The interaction scores were compared with progression free survival(PFS) of the patients (FIG. 14). Interaction scores of at least 803correlated well with survival. Notably, PD-L1 expression did notcorrelate with improved PFS (FIG. 15).

FIGS. 16 and 17 show a representative examples of overlaid masksindicating PD-L1-positive cells (red), PD-L-positive cells (yellow),tumor cells (S100, green), and all cells (DAPI, blue). For a positiveresponder to immunotherapy, the mask in FIG. 16 readily indicates thepresence of PD-L1-positive cells (red), PD-1-positive cells (yellow),and all tumor cells (green). In contrast, for a negative responder toimmunotherapy, the mask in FIG. 17 indicates the presence of tumor cells(S100, green) and all cells (DAPI, blue), but shows little to noPD-L1-positive cells (red) or PD-1-positive cells (yellow). FIG. 16represents an interaction score of 2176 (complete response toimmunotherapy). FIG. 17 represents an interaction score of 8 (noresponse to immunotherapy).

The tissue samples were also assessed using an FDA-approved method tomeasure PD-L1 in non-small cell lung cancer with the anti-PD-L1 antibodyclone 22C3, not currently used for melanoma tissue samples. PD-L1expression was compared with patient PFS and is shown in FIG. 19. Thismethod does not demonstrate statistically relevant diagnostic valuecompared to the methods described herein using interaction scores.

A verification cohort of 34 additional metastatic melanoma patients wasexamined and PD-1/PD-L1 interaction scores were obtained (see FIG. 20a). These interaction scores were also compared with progression freesurvival (PFS) of the patients (FIG. 20b ). Although not statisticallysignificant (p=0.19), the comparison indicates a trend of patients withhigher PD-1/PD-L1 interaction scores having longer PFS. Statisticalsignificance may be limited due to the relatively recent use of thesetherapies in the clinic therefore limiting the follow-up time. for thesepatients.

The PD-1/PD-L1 interaction scores as well as the comparison of thesescores with patient PFS or patient overall survival (OS) for thecombination of the earlier cohort of 26 patients with the verificationcohort of 34 patients are shown in FIGS. 20c-20e . Combined analysisclearly indicate patients with high PD-1/PD-L1 demonstrate an improvedresponse to anti-PD-1 therapies.

Example 2. Sample Preparation, Imaging, and Analysis of Imaging forNon-Small Cell Lung Carcinoma Tissue Samples from Human Patients

Analogous procedures as Example 1 were performed, substituting the mouseanti-S100 directly labeled with 488 dye with mouse anti-pan cytokeratindirectly labeled with 488 dye for epithelial tumor samples. Interactionscores for 38 samples are shown in FIG. 18.

Example 3. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing PD-L1 and Cells Expressing CD80

Sample Preparation

Formalin fixed paraffin embedded (FFPE) tissue samples were dewaxed,rehydrated and antigen retrieval was performed with elevated temperatureconditions. Staining was then performed where the following steps werecarried out. First, tissues were subjected to CTLA-4 expressiondetection using 20 pairs of hybridization probes spanning approximately1 kb of the CTLA-4 mRNA using RNAScope® (Advanced Cell Diagnostics). Insitu hybridization was visualized with TSA-Cy®3. The slides were washedand any residual HRP was then quenched using two washes of fresh 100 mMbenzhydrazide with 50 mM hydrogen peroxide. The slides were again washedbefore staining with a mouse anti-CD80 primary antibody. Slides werewashed and then incubated with an anti-mouse HRP secondary antibody.Slides were washed and then CD80 staining was detected using TSA-Cy® 5(Perkin Elmer). Any residual HRP was then quenched using two washes offresh 100 mM benzhydrazide with 50 mM hydrogen peroxide. The slides wereagain washed before staining with a rabbit anti-CD3 primary antibody.Slides were washed and then incubated with a cocktail of anti-rabbit HRPsecondary antibody plus 4′,6-diamidino-2-phenylindole (DAPI). Slideswere washed and then CD3 staining was detected using TSA-AlexaFluor488®(Life Technologies). Slides were washed a final time before they werecover-slipped with mounting media and allowed to dry overnight at roomtemperature.

Analogous imaging and analysis procedures as Example 1 were performed,imaging across DAPI, FITC, Cy® 3, and Cy® 5 wavelengths. Expression ofCTLA-4 and CD80 was used to develop an enrichment algorithm foracquiring 20× images. Analysis was performed to determine CTLA-4/CD80interaction scores by measuring the total area, in pixels, of CTLA-4 andCD3 positive cells within the proximity of CD80 positive cells dividedby the total area, in pixels, of the CD3 positive cells, multiplied by afactor of 10,000. Results are shown in FIG. 21.

Example 4. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing CTLA-4 and Cells Expressing CD80

Analogous procedures as Example 1 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof CTLA-4 and CD80.

Example 5. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing PD-L2 and Cells Expressing PD-1

Analogous procedures as Example 1 are performed, substituting thestaining and analysis of PD-L1 with the staining and analysis of PD-L2.

Example 6. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing CTLA-4 and Cells Expressing CD86

Analogous procedures as Example 1 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof CTLA-4 and CD86.

Example 7. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing LAG-3 and Cells Expressing HLA-DR

Analogous procedures as Example 1 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof LAG-3 and HLA-DR.

Example 8. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing TIM-3 and Cells Expressing Galectin9

Analogous procedures as Example 1 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof TIM-3 and Galectin 9.

Example 9. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing 41BB and Cells Expressing 4.1BBL

Analogous procedures as Example 1 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof 41BB and 4.1BBL.

Example 10. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing OX40 and Cells Expressing OX40L

Analogous procedures as Example 1 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof OX40 and OX40L.

Example 11. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing CD40 and Cells Expressing CD40L

Analogous procedures as Example 1 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof CD40 and CD40L.

Example 12. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing ICOS and Cells Expressing ICOSL

Analogous procedures as Example 1 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof ICOS and ICOSL.

Example 13. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing GITR and Cells Expressing GITRL

Analogous procedures as Example 1 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof GITR and GITRL.

Example 14. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing HLA-DR and Cells Expressing TCR

Analogous procedures as Example 1 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof HLA-DR and TCR.

Example 15. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing PD-1, PD-L1, and CD3

Analogous procedures as Example 1 were performed without the mouseanti-S100 antibody. Instead, after PD-L1 detection, primary andsecondary antibodies were removed via microwave. Slides were thenstained with rabbit anti-CD3 primary antibody. Slides were washed andthen incubated with a cocktail of anti-rabbit HRP secondary antibodyplus 4′,6-diamidino-2-phenylindole (DAPI). Slides were washed and thenCD3 staining was detected with TSA-AlexaFluor488 (Life Technologies).Imaging and analysis were analogous to Example 1 where the spatialproximity (e.g interaction score) was calculated by dividing the area ofPD-1 positive cells in the PD-L1 positive area, measured in pixels, bythe area of all nucleated cells, measured in pixels, multiplied by afactor of 10,000. Interaction scores for 29 samples are shown in FIG.22.

Manipulation of the image data from imaging device 100, for deriving apercent biomarker positivity value of the tissue sample, as describedherein, may be carried out by controller 1900, shown schematically inFIG. 23. Controller 1900 is shown to include a communications interface1902 and a processing circuit 1904. The processing circuit 1904 isconfigured to implement steps for determining percent biomarkerpositivity. It is contemplated that the elements and functions ofcontroller 1900 may be included in controller 150 of imaging device 100,or may be present in a computing system separate from imaging device100.

In some embodiments, the sample may be stained using a plurality offluorescence tags with affinity for specific biomarkers. A digital imageof the stained sample may be obtained, and the image further analyzedbased on the location of the fluorescence tags. Rather than whole-imageanalysis, fields of view may be prioritized based on the number of cellsthat express a first biomarker of interest. A predetermined number offields of view may then be further analyzed for fluorescence signals. Insome embodiments, the images of fluorescence signals are manipulated togenerate one or more masks of fluorescence signals corresponding tocells within the image. In some embodiments, the one or more masks offluorescence signals comprise one or more selected from the groupconsisting of a mask of all cells within the image, a mask of all cellsthat express the subset biomarker (e.g., all tumor cells), within theimage, a mask of all cells that do not express the subset biomarker(e.g., all non-tumor cells) within the image, a mask of all cellsexpressing a first biomarker of interest within the image, and abiomarker positive mask representing all cells of interest expressing abiomarker. The areas of these masks may be used to derive a value forPBP as desired. The total area of the biomarker mask may be used tocalculate a percent biomarker positivity value by dividing the area ofbiomarker mask for the cell type of interest by the area of all cells inthe compartment of interest.

FIG. 24 is a flowchart depicting the steps of one embodiment of a methodfor deriving a value for % biomarker positivity (PBP). In step 2001,image data is obtained and in step 2002, the image data is unmixed suchthat data specific to various types of fluorescence signals areseparated into different channels. In step 2003, data from a firstchannel is used to generate a mask of all cells. In step 2004, data froma second channel is used to generate a mask of the area in a field ofview that expresses a subset biomarker, for example, this subset maskmay be a mask of a tumor area present in a field of view. In step 2005,the all cell mask and the subset mask (e.g., a tumor area mask) arecombined to generate a mask of all subset cells. In certain embodiments,combining the all cell mask and the subset mask may identify all tumorcells and/or all non-tumor cells.

In some embodiments, a subset of cells identified by a subset biomarkerand a non-subset of cells corresponds to tumor cells and non-tumorcells, respectively or vice versa. In some embodiments, a subset ofcells identified by a subset biomarker and a non-subset of cellscorresponds to viable cells and non-viable cells, respectively or viceversa. In some embodiments, a subset of cells identified by a subsetbiomarker is a subset of viable cells and a non-subset of cells consistsof the viable cells not included in the subset of viable cells. In someembodiments, a subset of cells identified by a subset biomarker and anon-subset of cells corresponds to T cells and non-T cells, respectivelyor vice versa. In some embodiments, a subset of cells identified by asubset biomarker and a non-subset of cells corresponds to myeloid cellsand non-myeloid cells, respectively or vice versa.

The process may be carried out on only a selected type of cell ofinterest, for example, only tumor cells or only non-tumor cells. Theprocess may also be directed to an analysis of both. In step 2006, datafrom a third channel is used to generate a mask of all cells that arepositive for a biomarker (based on fluorescence signals representing thepresence of a fluorescent tag with an affinity for binding to theparticular biomarker of interest). In steps 2007 and 2008, the biomarkermask generated in step 2006 is combined with the subset cell maskgenerated in step 2005. Step 2007 combines the biomarker mask with thesubset cell mask in a first manner, to generate a mask of all subsetcells that are positive for the biomarker. Step 2008 combines thebiomarker mask with the subset cell mask, in a second manner, togenerate a mask of all subset cells that are not positive for thebiomarker. One or both of steps 2007 and 2008 may be performed accordingthe various embodiments of the method. In step 2009/2010, a PBP score iscalculated by dividing the area of the subset cells of interest (e.g.,the subset cells that are positive for the biomarker identified by themask created in step 2007 or the subset cells that are not positive forthe biomarker identified by the mask created in step 2008) by the totalarea of all cells of interest. Step 2009 calculates the PBP for subsetcells that are positive for the biomarker. Step 2010 calculates the PBPfor subset cells that are not positive for the biomarker. One or both ofsteps 2009 and 2010 may be performed according the various embodimentsof the method.

FIG. 25 is a flowchart depicting the steps of a second embodiment of amethod for deriving a value for % biomarker positivity (PBP). In step2101, image data is obtained and in step 2102, the image data is unmixedsuch that data specific to various types of fluorescence signals areseparated into different channels. In step 2103, data from a firstchannel is used to generate a mask of all cells. In step 2104, data froma second channel is used to generate a mask of all cells that arepositive for a biomarker (based on fluorescence signals representing thepresence of a fluorescent tag with an affinity for binding to theparticular biomarker of interest). In step 2105, a PBP score iscalculated by dividing the area of the cells that are positive for thebiomarker (which is identified by the mask created in step 2104) by thetotal area of all cells of interest (from step 2103). The process ofFIG. 25 may be carried out separately or concurrently with the methoddepicted in FIG. 24. In other words, a PBP score may be calculated forall cells, all tumor cells, and all non-tumor cells, or any combinationthereof, may combining the methods of FIGS. 24 and 25.

In FIG. 23, controller 1900 is shown to include a communicationsinterface 1902 and a processing circuit 1904. Communications interface1902 may include wired or wireless interfaces (e.g., jacks, antennas,transmitters, receivers, transceivers, wire terminals, etc.) forconducting data communications with various systems, devices, ornetworks. For example, communications interface 1902 may include anEthernet card and port for sending and receiving data via anEthernet-based communications network and/or a WiFi transceiver forcommunicating via a wireless communications network. Communicationsinterface 1902 may be configured to communicate via local area networksor wide area networks (e.g., the Internet, a building WAN, etc.) and mayuse a variety of communications protocols (e.g., BACnet, IP, LON, etc.).Communications interface 1902 may be a network interface configured tofacilitate electronic data communications between controller 1900 andvarious external systems or devices (e.g., imaging device 100). Forexample, controller 1900 may receive imaging data for the selectedfields of view from the imaging device 100, to analyze the data andcalculate the percent biomarker positivity

Processing circuit 1904 is shown to include a processor 1906 and memory1908. Processor 1906 may be a general purpose or specific purposeprocessor, an application specific integrated circuit (ASIC), one ormore field programmable gate arrays (FPGAs), a group of processingcomponents, or other suitable processing components. Processor 506 maybe configured to execute computer code or instructions stored in memory508 or received from other computer readable media (e.g., CDROM, networkstorage, a remote server, etc.).

Memory 1908 may include one or more devices (e.g., memory units, memorydevices, storage devices, etc.) for storing data and/or computer codefor completing and/or facilitating the various processes described inthe present disclosure. Memory 1908 may include random access memory(RAM), read-only memory (ROM), hard drive storage, temporary storage,non-volatile memory, flash memory, optical memory, or any other suitablememory for storing software objects and/or computer instructions. Memory1908 may include database components, object code components, scriptcomponents, or any other type of information structure for supportingthe various activities and information structures described in thepresent disclosure. Memory 508 may be communicably connected toprocessor 1906 via processing circuit 1904 and may include computer codefor executing (e.g., by processor 1906) one or more processes describedherein.

Still referring to FIG. 23, controller 1900 is shown to receive inputfrom an imaging device 100. The imaging device acquires all of theimaging data and records it, along with all of the meta-data whichdescribes it. The imaging device will then serialize the data into astream which can be read by controller 1900. The data stream mayaccommodate any binary data stream type such as the file system, a RDBMor direct TCP/IP communications. For use of the data stream, controller1900 is shown to include a spectral unmixer 1910. The spectral unmixer1910 may receive image data from an imaging device 100 on which itperforms spectral unmixing to unmix an image presenting variouswavelengths into individual, discrete channels for each band ofwavelengths. For example, the image data may be “unmixed” into separatechannels for each of the various fluorophores used to identify cells orproteins of interest in the tissue sample. The fluorophore, by way ofexample only, may be one or more of the group consisting of DAPI, Cy® 2,Cy® 3, Cy® 3,5, Cy® 5, FITC, TRITC, a 488 dye, a 555 dye, a 594 dye, andTexas Red. In one example, one of the channels may include image datathat falls within a predetermined band surrounding a wavelength of 461nm (the maximum emission wavelength for DAPI), to identify nuclei in theimage. Other channels may include image data for different wavelengthsto identify different portions of the tissue sample using differentfluorophores.

Controller 1900 is also shown to include various maskers, such as cellmasker 1912, subset area masker 1916, and biomarker masker 1922. These,or other maskers that may be included in the controller 1900 in otherembodiments, are used to receive an unmixed signal from the spectralunmixer 1910 and create a mask for the particular cell or area ofinterest, dependent on the fluorophore used to identify certain featuresof interest in the tissue sample. To create a mask, the maskers (such ascell masker 1912, subset area masker 1916, and biomarker masker 1922)receive image data related to an intensity of each pixel in the field ofview. Pixel intensity is directly proportional to the amount offluorescence emitted by the sample, which in turn, is directlyproportional to the amount of protein biomarker in the sample (whenusing a fluorophore to identify a particular biomarker). An absolutethreshold may be set based on the values which exist in the imagepixels. All the pixels which are greater than or equal to the thresholdvalue will be mapped to 1.0, or “on”, and all other pixels will bemapped to 0.0, or “off” In this way, a binary mask is created toidentify the cell or tissue portion of interest in the field of view. Inother embodiments, a mask is created using a lower bound wherein allpixels with an intensity at or above a lower bound are accepted and usedas the pixel value for the mask. If the intensity is below the lowerbound, the pixel value is set to 0.0, or “off”.

In the example flow diagram for masking shown in FIG. 26, it is shownthat the DAPI and 488 dye channels (or other fluorophore for identifyingnuclei and tumor areas, respectively) use the lower bound protocol(steps 2210, 2212, 2220, 2222), while the Cy5 channel (or otherfluorophore for identifying a biomarker of interest) uses a thresholdvalue protocol (step 2230), for providing the mask output. Inassociation with the lower bound protocol, there is also a histogramstep to determine the lower bound. In particular, histogram threshold(step 2212, 2222) produces a threshold of an input image but uses asliding scale to determine the point at which the thresholding occurs.The inputs are the current image and a user defined thresholdpercentage. The latter is used to determine at what percent of the totalintensity the threshold level should be set. Firstly, the intensity ofevery pixel is summed into a total intensity. The threshold percentageis multiplied by this total intensity to obtain a cut-off sum. Finally,all pixels are grouped by intensity (in a histogram) and theirintensities summed from lowest to highest (bin by bin) until the cut-offsum is achieved. The last highest pixel intensity visited in the processis the threshold for the current image. All pixels with intensitiesgreater than that value have their intensities set to maximum while allothers are set to the minimum.

The steps identified as steps 2214, 2216, 2224, 2226, 2228, 2232, 2234,2236 in FIG. 26 represent intermediary steps that occur in the initialmaskers, such as cell masker 1912 (steps 2214, 2216), subset area masker1916 (steps 2224, 2226, 2228), and biomarker masker 1922 (steps 2232,2234, 2236). These steps are defined as follows:

Dilate increases the area of brightest regions in an image. Two inputsare need for dilate. The first is the implicit current image and thesecond is the number of iterations to dilate. It is assumed that onlybinary images are used for the first input. The procedure will operateon continuous images, but the output will not be a valid dilate. Thedilate process begins by first finding the maximum pixel intensity inthe image. Subsequently, each pixel in the image is examined once. Ifthe pixel under investigation has intensity equal to the maximumintensity, that pixel will be drawn in the output image as a circle withiterations radius and centered on the original pixel. All pixels in thatcircle will have intensity equal to the maximum intensity. All otherpixels are copied into the output image without modification.

The fill holes procedure will fill “empty” regions of an image withpixels at maximum intensity. These empty regions are those that have aminimum intensity and whose pixel area (size) is that specified by theuser. The current image and size are the two inputs required. Likedilate this procedure should only be applied to binary images.

Erode processes images in the same fashion as dilate. All functionalityis the same as dilate except that the first step determines the minimumintensity in the image, only pixels matching that lowest intensity arealtered, and the circles used to bloom the found minimum intensitypixels are filled with the lowest intensity value. Like dilate thisprocedure should only be applied to binary images.

Remove Objects.

Two inputs are expected: the current image and object size. Removeobjects is the opposite of the fill holes procedure. Any regionscontaining only pixels with maximum intensity filling an area less thanthe input object size will be set to minimum intensity and thusly“removed.” This procedure should only be applied to binary images;application to continuous images may produce unexpected results.

The output at steps 2218, 2229, and 2238 are the resultant cell mask,subset mask (or, in this particular example, tumor area mask), andbiomarker cell mask, respectively. FIG. 26 further depicts thecombinations of these resultant masks to obtain the relevant areainformation for the PBP score. These combinations are described belowwith reference to the combination maskers of the controller 1900,depicted in FIG. 23.

Controller 1900 is shown to include combination maskers, such as subsetcell masker 1918, non-subset cell masker 1920, and combination masker1930. In some embodiments, the subset cells identified by masker 1918and the non-subset cells identified by masker 1920 are tumor cells andnon-tumor cells, respectively. In some such embodiments, the subset cellmasker 1918 is a tumor masker and the non-subset cell masker 1920 is anon-tumor masker. Subset cell masker performs an And operation, as shownat step 2252 in FIG. 26, to combine the output of the cell masker 1912(representative of all cells in the image) with the output of the subsetarea masker 1916. Accordingly, subset cell masker generates a mask ofall subset cells in the image. This same combination, using an Outoperation performed by non-subset cell masker 1920 as shown at step 2254in FIG. 26, generates a mask of all non-subset cells in the sampleimage.

Combination masker 1930 is configured to combine two input masks. Asdepicted in FIG. 26, combination masker 1930 combines the biomarker maskwith one of the subset cell mask (from subset cell masker 1918) ornon-subset cell mask (from non-subset cell masker 1920), or bothbiomarker mask+subset mask and biomarker mask+non-subset mask. Thedotted lines represent that either one or both of the cell masks may becombined with the biomarker mask at combination masker 1930. The resultof the combination masker 1930 is a mask representative of all subsetcells that are positive for the biomarker and/or all non-subset cellsthat are positive for the biomarker. The combination masker 1930 maycombine the masks in an alternate manner such that the result of thecombination masker 1930 is a mask representative of subset cells thatare not positive for the biomarker (i.e., biomarker negative). If thecells of interest are not specifically related to the subset, forexample tumor or non-tumor, but rather, all cells, then the biomarkerpositive mask is not combined with any additional mask and passesthrough the combination masker 1930 without modification.

To calculate the % biomarker positivity score (PBP), the area of theselected subset cell (e.g., all, tumor, or non-tumor) biomarker positivemask or biomarker negative mask (in which case the score representsbiomarker negativity) is determined in pixels at the area evaluator1932. The total area of all the selected cells (positive and notpositive for the biomarker), is determined in pixels at the areaevaluator 1932. The dotted lines terminating at area evaluator 1932indicate that the total area inputs may be one or more of the all cellmask, the subset cell mask, and the non-subset mask, to be calculatedseparately. A percent biomarker positivity score is determined at thepositivity calculator 1936. In one embodiment, the BPB score iscalculated by dividing the area of the selected cell biomarker positivemask from area evaluator 1932 by the area of the all selected cell maskfrom area evaluator 1932, and multiplying 100. In one embodiment, theequation executed by the interaction calculator 1936 is:

${BPB} = {\frac{A_{P}}{A_{A}} \times 100}$wherein A_(P) is a biomarker positive area for the selected type ofsubset cell (e.g., all, tumor, or non-tumor) and A_(A) is the total areaof all cells of the selected cell type (all, tumor, non-tumor). In someembodiments, A_(N) replaces A_(P) in the above equation, wherein A_(N)is a biomarker negative area for the selected type of cell (e.g., all,tumor, or non-tumor), to determine a score representative of percentbiomarker negativity for the type of subset cell.

The And procedure is modeled after a binary AND operation, but differsin significant ways. And accepts the current image and a user selectedresultant. The output is an image created by performing a multiplicationof the normalized intensities of matching pixels from the two inputimages. In some cases, image intensity data is already normalized.Therefore, the And procedure is simply a pixel-wise multiplication ofthe two images. The two inputs required for

Out are the current image and a user selected resultant. Out removes thesecond image form the first according to the formula A*(1−B/B_(max))where A is the current image, B the user selected image to remove, andB_(max) is the maximum intensity of B. Note that the division of B byB_(max) normalizes B.

In some embodiments, provided herein is an imaging system for deriving avalue for % biomarker positivity (PBP) for all cells or, optionally, oneor more subsets thereof, present in a field of view of a tissue sampletaken from a cancer patient, the imaging system comprising an imagingapparatus comprising a stage for positioning the sample in an imagingfield, an electromagnetic radiation source for directing electromagneticradiation at the sample, and a detector configured to detectelectromagnetic radiation from the sample, and a controller. Thecontroller comprises a user interface for exchanging information betweenan operator and the electronic control system and a processing circuitconfigured to execute instructions stored on a computer-readable medium.The instructions cause the electronic control system of the imagingsystem to: (i) receive information about the detected electromagneticradiation from the imaging apparatus; (ii) generate image data based onthe detected electromagnetic radiation; and (iii) analyze the image datato derive a value for PBP for all cells or, optionally, one or moresubsets thereof present in a field of view.

In some embodiments, analyzing the data comprises:

-   -   (i) generating an image of first fluorescence signals        representative of nuclei of all cells present in a field of        view, and dilating the first fluorescence signals to a diameter        of that of an entire cell to construct a first mask of all cells        present in the field of view;    -   (ii) constructing a second mask of second fluorescence signals        representative of all areas present in the field of view, which        express a subset biomarker;    -   (iii) optionally, constructing a third mask of third        fluorescence signals representative of all areas present in the        field of view, which express a first biomarker of interest;    -   (iv) combining said first and second masks in a manner that        provides a fourth mask comprising fluorescence signals        representative of all cells in the field of view, which also        express the subset biomarker;    -   (v) optionally, combining said first and third masks in a manner        that provides a fifth mask comprising fluorescence signals        representative of a first subset of all cells in the field of        view, which also express the first biomarker of interest;    -   (vi) deriving a value for PBP for all cells expressing the        subset biomarker by dividing the total area of the fourth mask        by the total area of the first mask;    -   (vii) optionally, combining said fourth and fifth masks in a        manner that provides a sixth mask comprising fluorescence        signals representative of the first subset of all cells in the        field of view, which express the subset biomarker and the first        biomarker of interest; and    -   (viii) optionally, deriving a value for PBP for the first subset        of all cells expressing the subset biomarker and the first        biomarker of interest by dividing the total area of the sixth        mask by the total area of the fourth mask.

In some embodiments, analyzing the data comprises:

-   -   (i) generating an image of first fluorescence signals        representative of nuclei of all cells present in a field of        view, and dilating the first fluorescence signals to a diameter        of that of an entire cell to construct a first mask of all cells        present in the field of view;    -   (ii) constructing a second mask of second fluorescence signals        representative of all areas present in the field of view, which        express a subset biomarker;    -   (iii) optionally, constructing a third mask of third        fluorescence signals representative of all areas present in the        field of view, which express a first biomarker of interest;    -   (iv) combining said first and second masks in a manner that        provides a fourth mask comprising fluorescence signals        representative of all cells in the field of view, which also        express the subset biomarker;    -   (v) optionally, combining said first and third masks in a manner        that provides a fifth mask comprising fluorescence signals        representative of all cells in the field of view, which also        express the first biomarker of interest;    -   (vi) deriving a value for PBP for all cells expressing the        subset biomarker by dividing the total area of the fourth mask        by the total area of the first mask;    -   (vii) optionally, combining said fourth and fifth masks in a        manner that provides a sixth mask comprising fluorescence        signals representative of all cells in the field of view, which        -   (a) express the subset biomarker and the first biomarker of            interest; or        -   (b) express the subset biomarker in the absence of the first            biomarker of interest;    -   and    -   (viii) optionally, deriving a value for PBP for the first subset        of all cells which either (a) express the subset biomarker and        the first biomarker of interest or (b) express the subset        biomarker in the absence of the first biomarker of interest, by        dividing the total area of the sixth mask by the total area of        the fourth mask.

In some embodiments, the optional steps are not performed. In someembodiments, all the recited optional steps are performed. In someembodiments, the total area is measured in pixels. In some embodiments,the total area of the fourth mask and the total area of the first maskare each measured in pixels. In some embodiments, the total area of thesixth mask and the total area of the fourth mask are each measured inpixels. In some embodiments, the total area of the first mask, the totalarea of the fourth mask, and the total area of the sixth mask are eachmeasured in pixels. In some embodiments, a pixel is 0.5 μm wide.

In some embodiments, analyzing the data further comprises:

-   -   (ix) constructing a seventh mask of fourth fluorescence signals        representative of all areas present in the field of view, which        express a second biomarker of interest;    -   (x) combining said first and seventh masks in a manner that        provides an eighth mask comprising fluorescence signals        representative of a second subset of all cells in the field of        view, which also express the second biomarker of interest;    -   (xi) combining said fourth and eighth masks in a manner that        provides a ninth mask comprising fluorescence signals        representative of the second subset of all cells in the field of        view, which express the subset biomarker and the second        biomarker of interest; and    -   (xii) deriving a value for PBP for the second subset of all        cells expressing the subset biomarker and the second biomarker        of interest by dividing the total area of the ninth mask by the        total area of the fourth mask.

In some embodiments, the total area is measured in pixels. In someembodiments, the total area of the ninth mask and the total area of thefourth mask are each measured in pixels. In some embodiments, a pixel is0.5 μm wide.

In some embodiments, analyzing the data further comprises:

-   -   (ix) constructing a seventh mask of fourth fluorescence signals        representative of all areas present in the field of view, which        express a second biomarker of interest;    -   (x) subtracting said second mask from said first mask in a        manner that provides an eighth mask comprising fluorescence        signals representative of all cells that do not express the        subset biomarker in the field of view;    -   (xi) combining said seventh and eighth masks in a manner that        provides a ninth mask comprising fluorescence signals        representative of all cells that express the second biomarker of        interest but do not express the subset biomarker in the field of        view; and    -   (xii) deriving a value for PBP for all cells that express the        second biomarker of interest but do not express the subset        biomarker by dividing the total area of the ninth mask by the        total area of the eighth mask.

In some embodiments, the method further comprises:

-   -   (ix) constructing a seventh mask of fourth fluorescence signals        representative of all areas present in the field of view, which        express a second biomarker of interest;    -   (x) combining said sixth and seventh masks in a manner that        provides an eighth mask comprising fluorescence signals        representative of all cells that        -   (a) express the subset biomarker, the first biomarker of            interest, and the second biomarker of interest in the field            of view;        -   (b) express the subset biomarker and the first biomarker of            interest in the absence of the second biomarker of interest            in the field of view; or        -   (c) express the subset biomarker and the second biomarker of            interest in the absence of the first biomarker of interest            in the field of view;    -   and    -   (xii) deriving a value for PBP for all cells that express the        first biomarker of interest or the second biomarker of interest,        or a combination thereof, as well as the subset biomarker, by        dividing the total area of the eighth mask by the total area of        the fourth mask.

In some embodiments, the method further comprises additional cycles ofsteps analogous to steps (ix), (x), and (xii) with respect to one ormore additional biomarkers of interest (e.g., a third biomarker ofinterest).

In some embodiments, the total area is measured in pixels. In someembodiments, the total area of the ninth mask and the total area of theeighth mask are each measured in pixels. In some embodiments, a pixel is0.5 μm wide.

In another aspect, provided herein are methods of deriving a value for %biomarker positivity (PBP) for all tumor cells present in a field ofview, comprising:

-   -   (i) generating an image of first fluorescence signals        representative of nuclei of all cells present in a field of        view, and dilating the first fluorescence signals to a diameter        of that of an entire cell to construct a first mask of all cells        present in the field of view;    -   (ii) constructing a second mask of second fluorescence signals        representative of all areas present in the field of view, which        express a tumor biomarker;    -   (iii) combining said first and second masks in a manner that        provides a third mask comprising fluorescence signals        representative of all tumor cells in the field of view;    -   (iv) constructing a fourth mask of third fluorescence signals        representative of all areas present in the field of view, which        express a biomarker of interest;    -   (v) combining said third and fourth masks in a manner that        provides a fifth mask comprising fluorescence signals        representative of all tumor cells in the field of view, which        also express the biomarker of interest; and    -   (vi) deriving a value for PBP for all tumor cells expressing the        biomarker of interest by dividing the total area of the fifth        mask by the total area of the third mask.

In some embodiments, the total area is measured in pixels. In someembodiments, the total area of the fifth mask and the total area of thethird mask are each measured in pixels. In some embodiments, a pixel is0.5 μm wide. In some embodiments, the biomarker of interest comprises abiomarker selected from the group consisting of PD-L1, Galectin 9, andMEW. In some embodiments, the biomarker of interest comprises PD-L1. Insome embodiments, the biomarker of interest comprises Galectin 9. Insome embodiments, the biomarker of interest comprises MEW. In someembodiments, the field of view further comprises non-tumor cells. Insome embodiments, the non-tumor cells comprise immune cells and stromalcells.

In another aspect, provided herein are methods of deriving a value for %biomarker positivity (PBP) for all non-tumor cells present in a field ofview, comprising:

-   -   (i) generating an image of first fluorescence signals        representative of nuclei of all cells present in a field of        view, and dilating the first fluorescence signals to a diameter        of that of an entire cell to construct a first mask of all cells        present in the field of view;    -   (ii) constructing a second mask of second fluorescence signals        representative of all areas present in the field of view, which        express a tumor biomarker;    -   (iii) subtracting said second mask from said first mask in a        manner that provides a third mask comprising fluorescence        signals representative of all non-tumor cells in the field of        view;    -   (iv) constructing a fourth mask of fluorescence signals        representative of all areas present in the field of view, which        express a biomarker of interest;    -   (v) combining said third and fourth masks in a manner that        provides a fifth mask comprising fluorescence signals        representative of all non-tumor cells in the field of view,        which also express the biomarker of interest; and    -   (vi) deriving a value for PBP for all non-tumor cells expressing        the biomarker of interest by dividing the total area of the        fifth mask by the total area of the third mask.

In some embodiments, the first subset of all the cells in the field ofview comprises tumor cells. In some embodiments, the first subset of allthe cells in the field of view comprises non-tumor cells. In someembodiments, the first subset of all the cells in the field of viewcomprises non-tumor and tumor cells.

In some embodiments, a subset of cells and a non-subset of cellscorresponds to tumor cells and non-tumor cells, respectively or viceversa. In some embodiments, a subset of cells and a non-subset of cellscorresponds to viable cells and non-viable cells, respectively or viceversa. In some embodiments, a subset of cells is a subset of viablecells and a non-subset of cells consists of the viable cells notincluded in the subset of viable cells. In some embodiments, a subset ofcells and a non-subset of cells corresponds to T cells and non-T cells,respectively or vice versa. In some embodiments, a subset of cells and anon-subset of cells corresponds to myeloid cells and non-myeloid cells,respectively or vice versa.

In some embodiments, the first subset of all the cells in the field ofview comprises T-cells. In some embodiments, the T-cells express CD3. Insome embodiments, the T-cells express CD8. In some embodiments, theT-cells express CD4.

In some embodiments the first subset of all the cells in the field ofview comprises myeloid cells. In further embodiments the myeloid cellsare myeloid derived suppressor cells. In further embodiments the myeloidcells are tumor associated macrophages. In some embodiments, the subsetbiomarker is expressed only in tumor cells. In some embodiments, thesubset biomarker is expressed only in non-tumor cells. In someembodiments, the subset biomarker is expressed in T-cells. In someembodiments, the subset biomarker comprises CD3. In some embodiments,the subset biomarker comprises CD19. In some embodiments, the subsetbiomarker is expressed in myeloid cells. In some embodiments, the subsetbiomarker is expressed in myeloid derived suppressor cells. In someembodiments, the subset biomarker is expressed in tumor associatedmacrophages. In some embodiments, the first biomarker of interestcomprises Ki67 and said first subset of all the cells in the field ofview comprises CD8 positive cells.

In some embodiments, the first biomarker of interest comprises abiomarker selected from the group consisting of CD11b, CD33, HLA-DR,IDO-1, ARG1, granzyme B, PD-L1, PD-L2, B7-H3, B7-H4, HLA-DR, Galectin 9,CD80, CD86, 4.1BBL, ICOSL, CD40, OX40L, GITRL, PD-1, TIM3, LAG3, 41BB,OX40, CTLA-4, CD40L, CD28, GITR, ICOS, CD28, CD3, CD4, CD8, FoxP3, CD25,CD16, CD56, CD68, CD163, CD80, and CD86. In some embodiments, the firstbiomarker of interest comprises a biomarker selected from the groupconsisting of PD-L1, PD-L2, B7-H3, B7-H4, HLA-DR, Galectin 9, CD80,CD86, 4.1BBL, ICOSL, CD40, OX40L, IDO-1, GITRL, ICOS, CD28, CD4, CD8,FoxP3, CD25, CD16, CD56, CD68, CD163, CD80, and CD86. In someembodiments, the first biomarker of interest comprises a biomarkerselected from the group consisting of PD-L1, PD-L2, B7-H3, B7-H4,HLA-DR, Galectin 9, CD80, CD86, 4.1BBL, ICOSL, CD40, OX40L, IDO-1, andGITRL. In some embodiments, the first biomarker of interest comprises abiomarker selected from the group consisting of PD-L1, Galectin 9, andMHC In some embodiments, the first biomarker of interest comprisesPD-L1.

In some embodiments, the second biomarker of interest comprises abiomarker selected from PD-1, TIM-3, and TCR. In some embodiments, thesecond biomarker of interest comprises PD-1.

In some embodiments, the first biomarker of interest and the secondbiomarker of interest are different from each other and comprise abiomarker selected from the group consisting of PD-L1, PD-L2, B7-H3,B7-H4, HLA-DR, Galectin 9, CD80, CD86, 4.1BBL, ICOSL, CD40, OX40L,IDO-1, GITRL, ICOS, CD28, CD3, CD4, CD8, FoxP3, CD25, CD16, CD56, CD68,CD163, CD80, and CD86. In some embodiments, the first biomarker ofinterest and the second biomarker of interest are different from eachother and comprise a biomarker selected from the group consisting ofCD11b, CD33, HLA-DR, ARG1, granzyme B, PD-L1, PD-L2, B7-H3, B7-H4,HLA-DR, Galectin 9, CD80, CD86, 4.1BBL, ICOSL, CD40, OX40L, IDO-1,GITRL, ICOS, CD28, CD3, CD4, CD8, FoxP3, CD25, CD16, CD56, CD68, CD163,CD80, and CD86.

In some embodiments, the first biomarker of interest comprises abiomarker selected from the group consisting of PD-L1, PD-L2, B7-H3,B7-H4, HLA-DR, Galectin 9, CD80, CD86, 4.1BBL, ICOSL, CD40, OX40L,GITRL, ICOS, CD28, CD4, CD8, FoxP3, CD25, CD16, CD56, CD68, CD163, CD80,and CD86; and the second biomarker of interest comprises a biomarkerselected from PD-1, TIM-3, and TCR. In some embodiments, the firstbiomarker of interest comprises PD-L1 and the second biomarker ofinterest comprises PD-1. In some embodiments, the first biomarker ofinterest comprises PD-L1 and the second biomarker of interest comprisesCD80. In some embodiments, the first biomarker of interest comprisesCTLA-4 and the second biomarker of interest comprises CD80. In someembodiments, the first biomarker of interest comprises PD-L2 and thesecond biomarker of interest comprises PD-1. In some embodiments, thefirst biomarker of interest comprises CTLA-4 and the second biomarker ofinterest comprises CD86. In some embodiments, the first biomarker ofinterest comprises LAG-3 and the second biomarker of interest comprisesHLA-DR. In some embodiments, the first biomarker of interest comprisesTIM-3 and the second biomarker of interest comprises Galectin 9. In someembodiments, the first biomarker of interest comprises 41BB and thesecond biomarker of interest comprises 4.1BBL. In some embodiments, thefirst biomarker of interest comprises OX40 and the second biomarker ofinterest comprises OX40L. In some embodiments, the first biomarker ofinterest comprises CD40 and the second biomarker of interest comprisesCD40L. In some embodiments, the first biomarker of interest comprisesICOS and the second biomarker of interest comprises ICOSL. In someembodiments, the first biomarker of interest comprises GITR and thesecond biomarker of interest comprises GITRL. In some embodiments, thefirst biomarker of interest comprises HLA-DR and the second biomarker ofinterest comprises TCR. In some embodiments, the first biomarker ofinterest comprises CD25 and the second biomarker of interest comprisesFoxP3. In some embodiments, the first biomarker of interest comprisesCD4 and the second biomarker of interest comprises CD8. In someembodiments, the first biomarker of interest comprises CD3 and thesecond biomarker of interest comprises PD-1. In some embodiments, thefirst biomarker of interest comprises CD56 and the second biomarker ofinterest comprises CD16. In some embodiments, the first biomarker ofinterest comprises HLA-DR and the second biomarker of interest comprisesIDO-1. In some embodiments, the first biomarker of interest comprisesCD33 and the second biomarker of interest comprises ARG1.

In some embodiments, the subset biomarker is only expressed in tumorcells. In some embodiments, the subset biomarker is expressed only innon-tumor cells. In some embodiments, the subset biomarker is expressedin T-cells. In some embodiments, the subset biomarker comprises CD3. Insome embodiments, the subset biomarker comprises CD19. In someembodiments, the subset biomarker comprises CD45. In some embodiments,the subset biomarker is expressed in myeloid cells. In some embodiments,the subset biomarker comprises CD11b.

In some embodiments, the first biomarker of interest comprises Ki67 andthe first subset of all the cells in the field of view comprises CD8positive cells.

In some embodiments, the non-tumor cells comprise immune cells andstromal cells. In some embodiments, the non-tumor cells comprise myeloidcells.

In some embodiments, the fluorescence signals are from four fluorescencetags, each specific to a different biomarker. In further embodiments, afirst fluorescence tag is associated with the first biomarker ofinterest, a second fluorescence tag is associated with the secondbiomarker of interest, a third fluorescence tag is associated with athird biomarker of interest, and a fourth fluorescence tag is associatedwith a fourth biomarker of interest. In some embodiments, the firstbiomarker of interest comprises a tumor and non-tumor marker. In someembodiments, the second biomarker of interest comprises a non-tumormarker. In some embodiments, the first biomarker of interest comprises atumor and non-tumor marker, and the second biomarker of interestcomprises a non-tumor marker. In some embodiments, the third biomarkerof interest is expressed by all cells. In some embodiments, the fourthbiomarker of interest is expressed only in tumor cells. In someembodiments, the third biomarker of interest is expressed by all cellsand the fourth biomarker of interest is expressed only in tumor cells.In some embodiments, the fourth biomarker of interest is the subsetbiomarker. In some embodiments, the third biomarker of interest isexpressed by all cells and the fourth biomarker of interest is thesubset biomarker. In some embodiments, one or more fluorescence tagscomprise a fluorophore conjugated to an antibody having a bindingaffinity for a specific biomarker or another antibody. In someembodiments, one or more fluorescence tags are fluorophores withaffinity for a specific biomarker.

In another aspect, disclosed are methods utilizing a system comprisingan imaging device and a controller for deriving a value for % biomarkerpositivity (PBP) for all cells or, optionally, one or more subsetsthereof present in a field of view of a tissue sample from a cancerpatient that are used in methods of treating cancer in the patient.

In some embodiments, disclosed herein are methods utilizing a systemcomprising an imaging device and a controller for treating cancer in apatient in need thereof, the method comprising deriving a value for %biomarker positivity (PBP) for all cells or, optionally, one or moresubsets thereof present in a field of view of a tissue sample from thepatient. In some embodiments, the deriving step is as described herein.

In some embodiments, disclosed herein are methods of deriving a valuefor % biomarker positivity (PBP) for all cells or, optionally, one ormore subsets thereof present in a field of view of a tissue samplecomprising: (i) using an imaging system to obtain image data for thetissue sample taken from a cancer patient, the imaging systemcomprising: a housing comprising a stage for positioning the sample inan imaging field, an electromagnetic radiation source for directingelectromagnetic radiation at the sample, and a detector for collectingelectromagnetic radiation output; and an electronic control systemcomprising memory and an processing circuit having image processingmodules; and (ii) analyzing, using the image processing modules, theimage data to derive the PBP for all cells or, optionally, one or moresubsets thereof present in a field of view. In some embodiments, themethods of deriving a value for PBP for all cells or, optionally, one ormore subsets thereof present in a field of view of a tissue sample areused as part of methods to monitor a progress of a patient diagnosedwith cancer and undergoing immunotherapy. In some embodiments, themethods of deriving a value for PBP for all cells or, optionally, one ormore subsets thereof present in a field of view of a tissue sample areused as part of methods to monitor immune cell modulation of a patientdiagnosed with cancer and undergoing immunotherapy.

Examples Example 16. Sample Preparation, Imaging, and Analysis ofImaging for Melanoma Tissue Samples from Human Patients

Sample Preparation.

Formalin fixed paraffin embedded (FFPE) tissue samples were dewaxed. Theslides were then rehydrated through a series of xylene to alcohol washesbefore incubating in distilled water. Heat-induced antigen retrieval wasthen performed using elevated pressure and temperature conditions,allowed to cool, and transferred to Tris-buffered saline. Staining wasthen performed where the following steps were carried out. First,endogenous peroxidase was blocked followed by incubation with aprotein-blocking solution to reduce nonspecific antibody staining. Next,the slides were stained with a mouse anti-PD1 primary antibody. Slideswere then washed before incubation with an anti-mouse HRP secondaryantibody. Slides were washed and then PD-1 staining was detected usingTSA+Cy® 3.5 (Perkin Elmer). Any residual HRP was then quenched using twowashes of fresh 100 mM benzhydrazide with 50 mM hydrogen peroxide. Theslides were again washed before staining with a rabbit anti-PD-L1primary antibody. Slides were washed and then incubated with a cocktailof anti-rabbit HRP secondary antibody plus mouse anti-S100 directlylabeled with 488 dye and 4′,6-diamidino-2-phenylindole (DAPI). Slideswere washed and then PD-L1 staining was detected using TSA-Cy® 5 (PerkinElmer). Slides were washed a final time before they were cover-slippedwith mounting media and allowed to dry overnight at room temperature. Aschematic overview of the antibodies and detection reagents is shown inFIG. 23.

Sample Imaging and Analysis.

Fluorescence images were then acquired using the Vectra 2 IntelligentSlide Analysis System using the Vectra software version 2.0.8 (PerkinElmer). First, monochrome imaging of the slide at 4× magnification usingDAPI was conducted. An automated algorithm (developed using inForm) wasused to identify areas of the slide containing tissue.

The areas of the slide identified as containing tissue were imaged at 4×magnification for channels associated with DAPI (blue), FITC (green),and Cy® 5 (red) to create RGB images. These 4× magnification images wereprocessed using an automated enrichment algorithm (developed usinginForm) in field of view selector 104 to identify and rank possible 20×magnification fields of view according to the highest Cy® 5 expression.

The top 40 fields of view were imaged at 20× magnification across DAPI,FITC, Texas Red, and Cy® 5 wavelengths. Raw images were reviewed foracceptability, and images that were out of focus, lacked any tumorcells, were highly necrotic, or contained high levels of fluorescencesignal not associated with expected antibody localization (i.e.,background staining) were rejected prior to analysis. Accepted imageswere processed using AQUAduct (Perkin Elmer), wherein each fluorophorewas spectrally unmixed by spectral unmixer 1910 into individual channelsand saved as a separate file.

The processed files were further analyzed using AQUAnalysis™ or througha fully automated process using AQUAserve™. Details were as follows.

Each DAPI image was processed by cell masker 1912 to identify all cellnuclei within that image (FIG. 7a ), and then dilated by 3 pixels torepresent the approximate size of an entire cell. This resulting maskrepresented all cells within that image (FIG. 7b ).

S100 (tumor cell marker for melanoma) detected with 488 dye (FIG. 8a )was processed by subset masker 1916 to create a binary mask of all tumorarea within that image (FIG. 8b ). Overlap between this mask and themask of all cells created a new mask for tumor cells (FIG. 8c ).

Similarly, absence of the tumor cell marker in combination with the maskof all nuclei created a new mask for all non-tumor cells (FIG. 8d ),performed by non-tumor cell masker 1920.

Each Cy® 5 image (FIG. 9a ) was processed by biomarker masker 1922 tocreate a binary mask of all cells that are PD-L1-positive (FIG. 9b ).Overlap between the mask of all tumor cells and the mask of allPD-L1-positive cells, using combination masker 1930, created a new maskof all PD-L1-positive tumor cells (FIG. 9c ). Similarly, overlap betweenthe mask of all non-tumor cells and the mask of all PD-L1-positivecells, using combination masker 1930, created a new mask of allPD-L1-positive non-tumor cells (FIG. 9d ).

Each Cy® 3.5 image (FIG. 10a ) was overlapped with the mask of allnon-tumor cells to create a binary mask of all cells that arePD-1-positive (FIG. 10b ).

The % biomarker positivity (PBP) for all tumor cells expressing PD-L1was derived, using positivity calculator 1936, by dividing the totalarea, measured in pixels and determined by area evaluator 1932, of themask of all PD-L1-positive tumor cells (FIG. 9c ) with the total area,measured in pixels and determined by area evaluator 1932, of the mask ofall tumor cells (FIG. 8c ). Representative values of PBP for all cellsexpressing PD-L1 are shown in FIG. 27a (data sorted according toincreasing expression).

The PBP for all non-tumor cells expressing PD-1 was derived by dividingthe total area, measured in pixels, of the mask of all PD-1-positivenon-tumor cells (FIG. 10b ) with the total area, measured in pixels, ofthe mask of all non-tumor cells (FIG. 8d ). Representative values of PBPfor all non-tumor cells expressing PD-1 are shown in FIG. 27b (datasorted according to increasing expression).

FIGS. 16 and 17 show a representative examples of overlaid masksindicating PD-L1-positive cells (red), PD-L-positive cells (yellow),tumor cells (S100, green), and all cells (DAPI, blue). FIG. 16 readilyindicates the presence of PD-L1-positive cells (red), PD-1-positivecells (yellow), and all tumor cells (green). In contrast, for a negativeresponder to immunotherapy, the mask in FIG. 17 indicates the presenceof tumor cells (S100, green) and all cells (DAPI, blue), but showslittle to no PD-L1-positive cells (red) or PD-1-positive cells (yellow).

Example 17. Sample Preparation, Imaging, and Analysis of Imaging forNon-Small Cell Lung Carcinoma (NSCLC) Tissue Samples from Human Patients

Analogous procedures as Example 15 were performed, substituting themouse anti-S100 directly labeled with 488 dye with a mouse anti-pancytokeratin directly labeled with 488 dye for epithelial tumor samples.PBP values for PD-1 and PD-L1 are shown in FIG. 18. A subset of thesepatients exhibited high levels of receptor-ligand interactionreminiscent of immune suppression.

Example 18. Comparison of Analysis Techniques

To generate control specimens, lymphoma cell lines with previouslyestablished expression of PD-L1 (Karpas 299) or lack of expression(Ramos RA #1) were cultured according to manufacturer's instructions.The cells were then counted and the two cell lines were mixed at varyingpercentages to generate a series of FFPE cell line pellet block rangingin PD-L1 expression from 0-100%. Cores (600 μm) from these cell linemixtures as well as from normal tonsil tissue resections were then usedto create a tissue microarray (TMA). A section of this TMA was thenstained, imaged and each core was then scored for % PD-L1 positivityusing AQUAnalysis™ (all steps as described in Example 15) and theresults are shown on the Y-axis of FIG. 38, where each point representsa single core (single field of view).

Alternatively, the same images were quantified for % PD-L1 expressionusing a cell counting based software for comparison as follows. DAPI wasused to first identify each cell nuclei and a morphological cytoplasmwas then created surrounding the identified cell nuclei. An intensitythreshold was established to identify cells with PD-L1 expression in thecytoplasm of the cells. The total number of cells identified above thisthreshold was then divided by the total number of cells in the image todetermine the % PD-L1 positive cells in each core and the results areshown on the X-axis of FIG. 28. Overall, there was a high level ofconcordance between the two methods of cell counting (R2=0.86, slope1.1); however, there were three noticeable outliers labeled as A, B, Cin FIG. 28 where the % PD-L1 positivity determined by AQUA® scoring wassignificantly higher than that of the cell counting method. Points A andB were cell line cores where 100% of the cells were Karpas299 and thusvalues determined by AQUA® scoring were much closer to expected and thecell counting method failed to identify the cytoplasm of the cells asPD-L1 positive. Similarly, in point C, the cell mixture included atheoretical 40% of Karpas299 cells where the value determined by AQUA®scoring was again much closer to expected over the cell counting method.These results demonstrated the superiority of methods disclosed hereinover the cell counting based software.

Example 19. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing PD-L1 and Cells Expressing CD80

Analogous procedures as Example 15 are performed, substituting thestaining and analysis of PD-1 with the staining and analysis of CD80.

Example 20. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing CTLA-4 and Cells Expressing CD80

Sample Preparation

Formalin fixed paraffin embedded (FFPE) tissue samples were dewaxed,rehydrated and antigen retrieval was performed with elevated temperatureconditions. Staining was then performed where the following steps werecarried out. First, tissues were subjected to CTLA-4 expressiondetection using 20 pairs of hybridization probes spanning approximately1 kb of the CTLA-4 mRNA using RNAScope® (Advanced Cell Diagnostics). Insitu hybridization was visualized with TSA-Cy®3. The slides were washedand any residual HRP was then quenched using two washes of fresh 100 mMbenzhydrazide with 50 mM hydrogen peroxide. The slides were again washedbefore staining with a mouse anti-CD80 primary antibody. Slides werewashed and then incubated with an anti-mouse HRP secondary antibody.Slides were washed and then CD80 staining was detected using TSA-Cy® 5(Perkin Elmer). Any residual HRP was then quenched using two washes offresh 100 mM benzhydrazide with 50 mM hydrogen peroxide. The slides wereagain washed before staining with a rabbit anti-CD3 primary antibody.Slides were washed and then incubated with a cocktail of anti-rabbit HRPsecondary antibody plus 4′,6-diamidino-2-phenylindole (DAPI). Slideswere washed and then CD3 staining was detected using TSA-AlexaFluor488®(Life Technologies). Slides were washed a final time before they werecover-slipped with mounting media and allowed to dry overnight at roomtemperature.

Analogous imaging and analysis procedures as Example 1 were performed,imaging across DAPI, FITC, Cy® 3, and Cy® 5 wavelengths. Expression ofCTLA-4 and CD80 was used to develop an enrichment algorithm foracquiring 20× images. Analysis was performed to survey the prevalence ofCTLA-4 in CD3 positive T cells and CD80 expression in tumor samplestaken from patients with metastatic melanoma. Results are shown in FIGS.44a and 44 b.

Example 21. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing PD-L2 and Cells Expressing PD-1

Analogous procedures as Example 15 are performed, substituting thestaining and analysis of PD-L1 with the staining and analysis of PD-L2.

Example 22. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing CTLA-4 and Cells Expressing CD86

Analogous procedures as Example 15 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof CTLA-4 and CD86.

Example 23. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing LAG-3 and Cells Expressing HLA-DR

Analogous procedures as Example 15 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof LAG-3 and HLA-DR.

Example 24. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing TIM-3 and Cells Expressing Galectin9

Analogous procedures as Example 15 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof TIM-3 and Galectin 9.

Example 25. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing 41BB and Cells Expressing 4.1BBL

Analogous procedures as Example 15 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof 41BB and 4.1BBL.

Example 26. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing OX40 and Cells Expressing OX40L

Analogous procedures as Example 15 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof OX40 and OX40L.

Example 27. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing CD40 and Cells Expressing CD40L

Analogous procedures as Example 15 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof CD40 and CD40L.

Example 28. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing ICOS and Cells Expressing ICOSL

Analogous procedures as Example 15 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof ICOS and ICOSL.

Example 29. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing GITR and Cells Expressing GITRL

Analogous procedures as Example 15 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof GITR and GITRL.

Example 30. Sample Preparation, Imaging, and Analysis of Imaging forTissue Samples with Cells Expressing HLA-DR and Cells Expressing TCR

Analogous procedures as Example 15 are performed, substituting thestaining and analysis of PD-L1 and PD-1 with the staining and analysisof HLA-DR and TCR.

Example 31. Sample Preparation, Imaging, and Analysis of CD3 and PD-1 onTissue Samples from Diffuse Large B-Cell Lymphoma (DLBCL) Patients

Sample preparation. Formalin fixed paraffin embedded (FFPE) tissuesamples from DLBCL patients (n=43) were dewaxed. The slides were thenrehydrated through a series of xylene to alcohol washes beforeincubating in distilled water. Heat-induced antigen retrieval was thenperformed using elevated pressure and temperature conditions, allowed tocool, and transferred to Tris-buffered saline. Staining was thenperformed where the following steps were carried out. First, endogenousperoxidase was blocked followed by incubation with a protein-blockingsolution to reduce nonspecific antibody staining. Next, the slides werestained with a mouse anti-PD1 primary antibody. Slides were then washedbefore incubation with an anti-mouse HRP secondary antibody. Slides werewashed and then PD-1 staining was detected using TSA+Cy® 3 (PerkinElmer). Primary and secondary antibody reagents were then removed viamicrowave. The slides were again washed before staining with a rabbitanti-CD3 primary antibody. Slides were washed and then incubated with acocktail of anti-rabbit HRP secondary antibody plus4′,6-diamidino-2-phenylindole (DAPI). Slides were washed and then CD3staining was detected using TSA-Cy® 5 (Perkin Elmer). Slides were washeda final time before they were cover-slipped with mounting media andallowed to dry overnight at room temperature. A schematic overview ofthe antibodies and detection reagents is shown in FIG. 31.

Sample Imaging and Analysis.

Fluorescence images were then acquired using the Vectra 2 IntelligentSlide Analysis System using the Vectra software version 2.0.8 (PerkinElmer). First, monochrome imaging of the slide at 4× magnification usingDAPI was conducted. An automated algorithm (developed using inForm) wasused to identify areas of the slide containing tissue.

The areas of the slide identified as containing tissue were imaged at 4×magnification for channels associated with DAPI (blue), Cy®3 (green),and Cy® 5 (red) to create RGB images. These 4× magnification images wereprocessed using an automated enrichment algorithm (developed usinginForm) in field of view selector 104 to identify and rank possible 20×magnification fields of view according to the highest Cy® 3 expression.

The top 40 fields of view were imaged at 20× magnification across DAPI,Cy®3, and Cy® 5 wavelengths. Raw images were reviewed for acceptability,and images that were out of focus, lacked any tumor cells, were highlynecrotic, or contained high levels of fluorescence signal not associatedwith expected antibody localization (i.e., background staining) wererejected prior to analysis. Accepted images were processed usingAQUAduct (Perkin Elmer), wherein each fluorophore was spectrally unmixedby spectral unmixer 210 into individual channels and saved as a separatefile.

The processed files were further analyzed using AQUAnalysis™ or througha fully automated process using AQUAserve™. Details were as follows.

Each DAPI image was processed by cell masker 1912 to identify all cellnuclei within that image (FIG. 29a ), and then dilated by 2 pixels torepresent the approximate size of an entire cell. This resulting maskrepresented all cells within that image (FIG. 29b ).

Each Cy® 5 image (FIG. 30a ) was processed by biomarker masker 1922 tocreate a binary mask of all cells that are PD-1-positive (FIG. 30b ).

Each Cy® 3 image (FIG. 31a ) was processed by biomarker masker 1922 tocreate a binary mask of all cells that are CD3-positive (FIG. 31b ).

The binary masks for all cells PD-1-positive and CD3-positive werecombined to create a binary mask of all cells that are double positivefor PD-1 and CD3 (FIG. 32).

The % biomarker positivity (PBP) for all CD3 cells expressing PD-1 wasderived, using positivity calculator 1936, by dividing the total area,measured in pixels and determined by area evaluator 1932, of the mask ofall PD-1-positive cells (FIG. 30b ) with the total area, measured inpixels and determined by area evaluator 1932, of the mask of allCD3-positive cells (FIG. 31b ). Differential distribution of exhaustedT-cells (CD3+/PD1+) were observed in primary (low levels) versussecondary sites (high levels). Results are shown in FIGS. 33a and 33 b.

Example 32. Platform Comparison

The accuracy of analogous procedures as Examples 15 and 30 was confirmedby comparison with flow cytometry. The frequencies of regulatory T-cells(based on FoxP3 and CD25 expression) was ascertained in whole bloodstimulated with IL-2, TGF β, and CD28 for 5 days on a CD3-coated plate.

Platform FoxP3 CD25 Flow cytometry 22% 66% PBP method 24% 66%

Example 33. Assessment of CD25/FoxP3 T-Cells in Multiple TumorIndications

Analogous procedures as Example 30 were performed with the additionalidentification of tumor cells with either anti-S100 or anti-cytokeratinantibodies detected with an AlexaFluor488 secondary antibody and imagedacross DAPI, FITC, Cy® 3, and Cy® 5 wavelengths for quantitativeassessment of CD25 and FoxP3 on NSCLC, gastric, and melanoma tissues.The tissues were stained with antibodies recognizing CD25 and FoxP3 andtheir expression in non-tumor areas were calculated as % expression. Theprevalence of CD25/FoxP3+ T-cells ranged from 1% to 10% in archivallung, gastric, and melanoma tissue specimens. Results are shown in FIGS.34 and 35.

Example 34. Assessment of CD4/CD8 T-Cells in Multiple Tumor Indications

Analogous procedures as Example 30 were performed with the additionalidentification of tumor cells with either anti-S100 or anti-cytokeratinantibodies detected with an AlexaFluor488 secondary antibody and imagedacross DAPI, FITC, Cy® 3, and Cy® 5 wavelengths for quantitativeassessment of CD4 and CD8 on NSCLC, gastric, and melanoma tissues. Thetissues were stained with antibodies recognizing CD4 and CD8 and theirexpressions in non-tumor areas were calculated as % expression. A broadrange of expression (10%-50%) was observed for CD4+ and CD8+ T-cells insequential sections of the tumor specimens. Results are shown in FIGS.36a and 36 b.

Example 35. Assessment of Myeloid Derived Suppressor Cell (MDSC)-LikeCells in Tumor Samples from Patients Diagnosed with Metastatic Melanomaor Non-Small Cell Lung Cancer

To identify MDSC-like cells expressing phenotypic markers characteristicof myeloid cells (e.g., CD11b, CD33, and HLA-DR) and biochemical markers(e.g., ARG1 and IDO-1) that render suppressive function upon thesecells, samples were stained with either CD11b, HLA-DR, and IDO, orCD11b, CD33, and ARG1.

Differential expression of CD11b, HLA-DR and IDO-1 was utilized tosurvey presence of a subset of suppressive myeloid cells known as TAMs(tumor associated macrophages) in tumor biopsies from metastaticmelanoma patients. Representative sub-phenotypes that may be relevantfor predicting response to cancer immunotherapies are shown in FIGS.37a, 37b, 38a , and 38 b.

Differential expression of CD11b, CD33, and ARG-1 or CD11b, HLA-DR, andIDO-1 were utilized to survey presence of MDSC like cells and TAMs intumor specimens from advanced lung cancer (NSCLC) patients.Representative sub-phenotypes that may be relevant for predictingresponse to cancer immunotherapies are shown in FIGS. 39a, 39b , and 40.

Sample Preparation.

Formalin fixed paraffin embedded (FFPE) tissue samples were dewaxed. Theslides were then rehydrated through a series of xylene to alcohol washesbefore incubating in distilled water. Heat-induced antigen retrieval wasthen performed using elevated pressure and temperature conditions,allowed to cool, and transferred to Tris-buffered saline. Staining wasthen performed where the following steps were carried out. First,endogenous peroxidase was blocked followed by incubation with aprotein-blocking solution to reduce nonspecific antibody staining. Next,the slides were stained with either a rabbit anti-IDO-1 or mouseanti-CD33 primary antibody. Slides were then washed before incubationwith anti-rabbit or anti-mouse HRP secondary antibody. Slides werewashed and then anti-IDO-1 or anti-CD33 staining was detected usingTSA+Cy® 5 (Perkin Elmer). Any residual HRP was then quenched using twowashes of fresh 100 mM benzhydrazide with 50 mM hydrogen peroxide. Theslides were again washed before staining with a mouse anti-HLA-DR or arabbit anti-ARG1 primary antibody. Slides were washed and then incubatedwith anti-mouse or anti-rabbit HRP secondary antibody. Slides werewashed and then the anti-HLA-DR or anti-ARG1 staining was detected usingTSA-Cy® 3 (Perkin Elmer). Primary and secondary antibody reagents werethen removed via microwave. The slides were again washed before stainingwith a rabbit anti-CD11b antibody. Slides were washed and then incubatedwith a cocktail of anti-rabbit HRP secondary antibody plus4′,6-diamidino-2-phenylindole (DAPI). Slides were washed and thenanti-CD11b staining was detected using TSA-AlexaFluor488 (LifeTechnologies). Slides were washed a final time before they werecover-slipped with mounting media and allowed to dry overnight at roomtemperature.

Analogous procedures to Example 16 were used for sample imaging andanalysis across DAPI, FITC, Cy®3, and Cy®5 wavelengths. 4× magnificationimages were processed using an automated enrichment algorithm (developedusing inForm) in field of view selector 104 to identify and rankpossible 20× magnification fields of view according to the highest Cy® 3and Cy® 5 expression.

Each DAPI image was processed by cell masker 212 to identify all cellnuclei within that image and then dilated to represent the approximatesize of an entire cell. This resulting mask represented all cells withinthat images

Each AlexaFluor488® image was processed by biomarker masker 222 tocreate a binary mask of all cells that are CD11b positive.

Each Cy® 3 image was processed by biomarker masker 222 to create abinary mask of all cells that are HLA-DR or CD33 positive.

Each Cy® 5 image was processed by biomarker masker 222 to create abinary mask of all cells that are IDO-1 or ARG1 positive.

The binary masks for all cells CD11b positive and HLA-DR positive werecombined to create a binary mask of all cells that are either doublepositive for CD11b and HLA-DR or are CD11b positive and HLA-DR negative.

The % biomarker positivity (PBP) for all CD11b cells lacking expressionof HLA-DR was derived, using positivity calculator 236, by dividing thetotal area, measured in pixels and determined by area evaluator 232, ofthe mask of all CD11b-positive, HLA-DR-negative cells with the totalarea, measured in pixels and determined by area evaluator 232, of themask of all CD11b-positive cells. Results are shown in FIG. 37a fortumor samples obtained from patients diagnosed with metastatic melanoma.

The binary masks for all cells CD11b positive, IDO positive, and HLA-DRpositive were combined to create a binary mask of all cells that areCD11b-positive, HLA-DR-negative, and IDO-1-positive.

The PBP for all CD11b cells expressing IDO-1, but lacking expression ofHLA-DR was derived by dividing the total area, measured in pixels, ofthe mask of all CD11b-positive, HLA-DR-negative, IDO-1-positive cellswith the total area, measured in pixels, of the mask of allCD11b-positive cells. Results are shown in FIG. 37b for tumor samplesobtained from patients diagnosed with metastatic melanoma and FIG. 40for patients diagnosed with non-small cell lung cancer.

The binary masks for all cells HLA-DR positive and IDO-1 positive werecombined to create a binary mask of all cells that are double positivefor HLA-DR and DO-1.

The % biomarker positivity (PBP) for all HLA-DR cells expressing IDO-1was derived, using positivity calculator 236, by dividing the totalarea, measured in pixels and determined by area evaluator 232, of themask of all IDO-1-positive, HLA-DR-positive cells with the total area,measured in pixels and determined by area evaluator 232, of the mask ofall HLA-DR-positive cells. Results are shown in FIG. 38a for tumorsamples obtained from patients diagnosed with metastatic melanoma.

The binary masks for all cells CD11b positive, IDO positive, and HLA-DRpositive were combined to create a binary mask of all cells that areCD11b-positive, HLA-DR-positive, and IDO-1-positive.

The PBP for all CD11b cells expressing IDO-1 and HLA-DR was derived bydividing the total area, measured in pixels, of the mask of allCD11b-positive, HLA-DR-positive, IDO-1-positive cells with the totalarea, measured in pixels, of the mask of all CD11b-positive cells.Results are shown in FIG. 38b for tumor samples obtained from patientsdiagnosed with metastatic melanoma and FIG. 39b for tumor samplesobtained from patients diagnosed with non-small cell lung cancer.

The binary masks for all cells CD11b positive, CD33 positive, and ARG1positive were combined to create a binary mask of all cells that areCD11b-positive, CD33-positive, and ARG1-positive.

The PBP for all CD11b cells expressing CD33 and ARG1 was derived bydividing the total area, measured in pixels, of the mask of allCD11b-positive, CD33-positive, ARG1-positive cells with the total area,measured in pixels, of the mask of all CD11b-positive cells. Results areshown in FIG. 39a for tumor samples obtained from patients diagnosedwith non-small cell lung cancer.

Example 36. Assessment of Activated T Cells in Tumor Samples fromPatients Diagnosed with Metastatic Melanoma

Analogous procedures as Example 20 were performed to stain each samplewith a combination of DAPI, CD3, CD8, and Ki67 to identifysub-populations of activated T cells. Prevalence of CD8+Ki67+ wassurveyed in tumor biopsies obtained from patients diagnosed withmetastatic melanoma (FIG. 41).

Example 37. Assessment of Macrophage Prevalence in Tumor Samples fromPatients Diagnosed with Metastatic Melanoma

Analogous procedures as Example 16 were performed with the additionalidentification of tumor cells with anti-S100 antibody detected with anAlexaFluor488 secondary antibody and imaged across DAPI, FITC, Cy® 3,and Cy® 5 wavelengths for quantitative assessment of CD163 and CD68 onmelanoma tissues. The tissues were stained with antibodies recognizingCD163 and CD68 and their expression in non-tumor areas were calculatedas single PBP expression or double PBP expression. Results are shown inFIGS. 42a, 42b , and 42 c.

Example 38. Assessment of T Cell Suppression Prevalence in Tumor Samplesfrom Patients Diagnosed with Diffuse Large B-Cell Lymphoma (DLBCL) andNeuroendocrine Tumors (NET)

Analogous procedures as Example 1 were performed to stain the DLBCL andNET tumor specimens with a mouse anti-LAG-3 primary antibody, anti-mouseHRP secondary, detected with TSA+Cy®3.5, with remaining HRP quenchedwith 100 mM benzhydrazide and 50 mM hydrogen peroxide. Following this,slides were stained with a rabbit anti-TIM-3 primary antibody,anti-rabbit HRP secondary, detected with TSA-Cy®5. Primary and secondaryantibodies were then removed via microwave. Tissues were then stainedwith a rabbit anti-CD3 primary antibody, anti-rabbit HRP secondary plus4′,6-diamidino-2-phenylindole (DAPI), detected with Opal™ 520. Imagingwas performed analogous to Example 1 across DAPI, FITC, Texas Red, andCy® 5 wavelengths. Analysis was performed analogous to Example 1 todetermine PBP prevalence of T cells that were LAG-3 and TIM-3 positiverespectively. Results are shown in FIGS. 43a and 43 b.

Para. A. An image processing system, comprising:

-   -   an image processing controller comprising a processing circuit        configured to execute instructions stored on a computer-readable        medium which cause the controller of the image processing system        to: receive image data from an imaging device, the image data        describing a sample of tumor tissue taken from a cancer patient;        and    -   determine, using the image data, a score representative of a        nearness between at least one pair of cells in the tumor tissue,        a first member of the at least one pair of cells expressing a        first biomarker and a second member of the at least one pair of        cells expressing a second biomarker that is different from the        first biomarker;    -   wherein the score is indicative of a likelihood that the cancer        patient will respond positively to immunotherapy.

Para. B. The system of Para. A in which the score representative of anearness between at least one pair cells is representative of an extentthat the pair of cells are within a predetermined proximity of oneanother.

Para. C. The system of Para. A in which the nearness is assessed on apixel scale.

Para. D. The system of Para. B in which the predetermined proximitybetween the pair of cells ranges from about 1 pixel to about 100 pixels.

Para. E. The system of Para. B in which the predetermined proximitybetween the pair of cells ranges from about 5 pixel to about 40 pixels.

Para. F. The system of Para. B in which the predetermined proximitybetween the pair of cells ranges from about 0.5 μm to about 50 μm.

Para. G. The system of Para. B in which the predetermined proximitybetween the pair of cells ranges from about 2.5 μm to about 20 μm.

Para. H. The system of Para. A in which the score is determined byobtaining a proximity between the boundaries of the pair of cells.

Para. I. The system of Para. A in which the score is determined byobtaining a proximity between the centers of mass of the pair of cells.

Para. J. The system of Para. A in which the score is determined usingboundary logic based on a perimeter around a selected first cell of thepair of cells.

Para. K. The system of Para. A in which the score is determined bydetermining an intersection in the boundaries of the pair of cells.

Para. L. The system of Para. A in which the score is determined bydetermining an area of overlap of the pair of cells.

Para. M. The system of Para. A in which the processing circuit isfurther configured to execute instructions which cause the controller ofthe image processing system to:

separate the image data into unmixed image data; and

provide the data through a plurality of data channels, in which theunmixed image data in a first data channel describes fluorescencesignals attributable to the first biomarker and the unmixed image datain a second data channel describes fluorescence signals attributable tothe second biomarker.

Para. N. The system of Para. M in which, to determine the score, theprocessing circuit is further configured to execute instructions whichcause the controller of the image processing system to: dilatefluorescence signals attributable to the first biomarker from the firstdata channel by a predetermined margin that is selected to encompassproximally located cells expressing the second biomarker to generate adilated first biomarker mask;

determine an interaction area, wherein the interaction area is a firsttotal area for all cells which express the second biomarker and areencompassed within the dilated fluorescence signals attributable to thecells expressing the first biomarker; and

divide the interaction area by a normalization factor, and multiplyingthe resulting quotient by a predetermined factor to arrive at a spatialproximity score.

Para. O. The system of Para. N in which the normalization factor is atotal area for all cells that have a capacity to express the secondbiomarker.

Para. P. The system of Para. N in which the interaction area isdetermined by combining the dilated first biomarker mask with a maskrepresentative of cells that express the second biomarker, determinedfrom signals of the second data channel.

Para. Q. The system of Para. M in which a third data channel describesfluorescence signals attributable to cell nuclei and a fourth datachannel describes fluorescence signals attributable to tumor area in thesample.

Para. R. The system of Para. O in which the total area for all cellsthat have a capacity to express the second biomarker is determined bycombining a cell mask representative of all cells in the sample, basedon signals from the third data channel, and a tumor area maskrepresentative of the tumor area on the sample, based on signals fromthe fourth data channel.

Para. S. The system of Para. R in which combining the cell mask and thetumor area mask comprises removing the tumor area mask from the cellmask.

Para. T. The system of any of Paras. N-S in which each mask is generatedby assigning a binary value to each pixel of image data from a selectedchannel.

Para. U. The system of Para. T in which the binary value is assigned toeach value by a threshold function, wherein the threshold functionassigns a value of 1 to each pixel that has an intensity equal to orgreater than a predetermined intensity threshold, and assigns a value of0 to each pixel that has an intensity below the predetermined intensitythreshold.

Para. V. The system of Para. T in which the binary value is assigned toeach value by a histogram threshold function, wherein the histogramthreshold function uses a sliding scale of pixel intensity to determinea threshold, and assigns a value of 1 to each pixel that has anintensity equal to or greater than an intensity threshold, and assigns avalue of 0 to each pixel that has an intensity below the intensitythreshold.

Para. W. The system of Para. V in which the threshold is determined by:

summing an intensity of every pixel into a total intensity;

multiplying a threshold percentage by the total intensity to obtain acut-off sum;

grouping all pixels by intensity in a histogram; and

summing the pixel intensities from lowest to highest until the cut-offsum is achieved;

wherein the last highest pixel intensity visited in the process is theintensity threshold.

Para. X. The system of any of Paras. N-W in which combining the masks isperformed using an “and” operation; wherein the “and” operation is amultiplication of the pixel intensity of a pixel in a first mask by thepixel intensity of the pixel in a second mask.

Para. Y. The system of any of Paras. N-W in which combining the masks isperformed using an “out” operation; wherein the “out” operation removesthe a second mask from a first mask.

Para. Z. The system of Para. A in which the processing circuit isfurther configured to execute instructions which cause the controller ofthe image processing system to:

obtain image data at a low magnification representative of theconcentration of the first or the second biomarker in the image;

identify areas that include the highest concentration of the first orthe second biomarker;

select a predetermined number of the areas including the highestconcentration of the first or the second biomarker;

send instructions to imaging device to obtain high magnification imagedata for the predetermined number of areas; and

wherein the high magnification image data is provided to the controllerto be analyzed and used to determine the score.

Para. AA. The system of Para. Z in which the low magnification is lessthan or equal to 10× magnification and wherein the high magnification isgreater than 10×.

Para. AB. The system of Para. AA in which the low magnification is 10×magnification and wherein the high magnification is 40×.

Para. AC. The system of Para. AA in which the low magnification is 10×magnification and wherein the high magnification is 20×.

Para. AD. The system of Para. AA in which the low magnification is 4×magnification and wherein the high magnification is 40×.

Para. AE. The system of Para. AA in which the low magnification is 4×magnification and wherein the high magnification is 20×.

Para. AF. The system of Para. A wherein the image processing unit isfurther configured to associate the score with metadata associated withthe images of the sample.

Para. AG. The system of Para. A wherein the image processing unitgenerates a report including the score.

Para. AH. The system of Para. A wherein the image processing unit isfurther configured to provide the score to an operator to determineimmunotherapy strategy.

Para. AI. The system of Para. A wherein the image processing unit isfurther configured to record the score in a database.

Para. AJ. The system of Para. A wherein the image processing unit isfurther configured to associate the score with a patient's medicalrecord.

Para. AK. The system of Para. A wherein the image processing unit isfurther configured to display the score on a display device.

Para. AL. The system of Para. A in which the first member of the atleast one pair of cells comprises a tumor cell and the second member ofthe at least one pair of cells comprises a non-tumor cell.

Para. AM. The system of Para. AL in which the non-tumor cell comprisesan immune cell.

Para. AN. The system of Para. A in which the first and second members ofthe at least one pair of cells comprise immune cells.

Para. AO. The system of Para. A in which the first member of the atleast one pair of cells comprises a tumor cell, a myeloid cell, or astromal cell and the second member of the at least one pair of cellscomprises an immune cell.

Para. AP. The system of Para. AO in which the tumor cell, myeloid cell,or stromal cell expresses PD-L1 and the immune cell expresses PD-1.

Para. AQ. The system of Para. A in which the image processing unit isfurther configured to select a predetermined number of fields of viewavailable from the sample comprising tumor tissue taken from the cancerpatient and determine the score from data associated with each field ofview.

Para. AR. The system of Para. AQ in which the sample is stained with aplurality of fluorescence tags, and wherein each of the fluorescencetags is directed to a specific biomarker.

Para. AS. The system of Para. AQ or Para. AR, in which the plurality offluorescence tags comprises a first fluorescence tag for the firstbiomarker and a second fluorescence tag for the second biomarker.

Para. AT. The system of any one of Paras. AQ-AS in which the marginranges from about 1 to about 100 pixels.

Para. AU. The system of any one of Paras. AQ-AT in which the proximallylocated cells expressing the second biomarker are within about 0.5 toabout 50 μm of a plasma membrane of the cells that express the firstbiomarker.

Para. AV. The system of any one of Paras. N-AU in which thepredetermined factor is 10⁴.

Para. AW. The system of Para. A in which the first member of the atleast one pair of cells expresses a first biomarker selected from thegroup consisting of PD-L1, PD-L2, B7-H3, B7-H4, HLA-DR, Galectin 9,CD80, CD86, 4.1BBL, ICOSL, CD40, OX40L, IDO-1, GITRL, and combinationsthereof, and the second member of the at least one pair of cellsexpresses a second biomarker selected from the group consisting of PD-1,TIM3, LAG3, 41BB, OX40, CTLA-4, CD40L, CD28, GITR, ICOS, CD28, andcombinations thereof.

Para. AX. The system of any one of Paras. A-AW in which the thresholdvalue ranges from about 500 to about 5000.

Para. AY. The system of Para. AX in which the threshold value is about900 plus or minus 100.

Para. AZ. The system of any one of Paras. A-AY in which theimmunotherapy comprises immune checkpoint therapy.

Para. BA. A system for determining a score representative of a proximitybetween at least one pair of cells in a tumor tissue sample, the systemcomprising:

a processing circuit comprising a memory, the memory comprising:

-   -   a spectral unmixer configured to receive image data from an        imaging device, separate the image data into unmixed image data,        and provide the data through a plurality of data channels, in        which the unmixed image data in a first data channel describes        fluorescence signals attributable to a cell nucleus, the unmixed        data in a second data channel describes fluorescence signals        attributable to tumor tissue, the unmixed data in a third data        channel describes fluorescence signals attributable to a first        biomarker, and the unmixed image data in a fourth data channel        describes fluorescence signals attributable to a second        biomarker;    -   a cell masker configured to receive image data from the first        data channel and generating an image representative of all        nuclei in a field of view and dilating the image to generate a        cell mask;    -   a tumor area masker configured to receive image data from the        second data channel and generating a mask of all tumor area in        the field of view;    -   a first biomarker masker configured to receive image data from        the third data channel and generating a first biomarker mask of        all cells in the field of view that express the first biomarker;    -   a second biomarker masker configured to receive image data from        the fourth data channel and generating a second biomarker mask        of all cells in the field of view that express the second        biomarker;    -   a tumor masker configured to combine the cell mask and the tumor        area mask to generate at least one of a non-tumor cell mask and        a tumor cell mask;    -   a dilator configured to dilate the first biomarker mask to        generate a dilated first biomarker mask;    -   an interaction masker configured to the dilated first biomarker        mask and the second biomarker mask to generate an interaction        mask;    -   an interaction calculator configured to calculate a spatial        proximity score.

Para. BB. The system of Para. BA in which the interaction calculator isconfigured to divide the area of the interaction mask by the area of oneof the non-tumor cell mask, the tumor cell mask, and the cell mask.

Para. BC. The system of Para. BA in which the spatial proximity score iscalculated according to the equation:

${SPS} = {\frac{A_{I}}{A_{C}} \times 10^{4}}$wherein A_(I) is a total interaction area (area of the interaction mask)and A_(C) is the total area of the at least one of the non-tumor cellmask, the tumor cell mask, and the cell mask.

Para. BD. A method of processing an image of a tissue sample todetermine a score representative of a proximity between at least onepair of cells in the tissue, the method comprising:

receiving image data associated with a tissue sample comprising tumortissue taken from the cancer patient;

selecting at least one field of view from the sample which is stainedwith a plurality of fluorescence tags, which selection is biased towardselecting at least one field of view that contains a greater number ofcells that express the first biomarker relative to other fields of view;

dilating, for each of the selected fields of view, fluorescence signalsattributable to the first biomarker by a margin sufficient to encompassproximally located cells expressing the second biomarker;

dividing a first total area for all cells from each of the selectedfields of view, which express the second biomarker and are encompassedwithin the dilated fluorescence signals attributable to the cellsexpressing the first biomarker, with a normalization factor, andmultiplying the resulting quotient by a predetermined factor to arriveat a spatial proximity score; and

recording the score, which score when compared to a threshold value isindicative of a likelihood that the cancer patient will respondpositively to immunotherapy.

Para. BE. The system of Para. A, wherein the method provides a superiorpredictive power compared to a quantitation of expression of the firstbiomarker or a quantitation of expression of the second biomarker.

Para. BF. The system of Para. BA, wherein the method provides a superiorpredictive power compared to a quantitation of expression of the firstbiomarker or a quantitation of expression of the second biomarker.

Para. BG. The method of Para. BD, wherein the method provides a superiorpredictive power compared to a quantitation of expression of the firstspecific biomarker or a quantitation of expression of the secondspecific biomarker.

Para. BH. The method of Para. BG or the system of Para. BE or Para. BF,wherein the predictive power is quantified as a positive predictivevalue, a negative predictive value, or a combination thereof.

Para. BI. The method or system of Para. BH, wherein the positivepredictive value is 65% or greater.

Para. BJ. The method or system of Para. BI, wherein the positivepredictive value is 70% or greater.

Para. BK. The method or system of Para. BJ, wherein the positivepredictive value is 75% or greater.

Para. BL. The method or system of Para. BH, wherein the negativepredictive value is 65% or greater.

Para. BM. The method or system of Para. BL, wherein the negativepredictive value is 80% or greater.

Para. BN. A system for determining a value of biomarker positivity in atumor tissue sample, the system comprising:

a processing circuit comprising a memory, the memory comprising:

-   -   a spectral unmixer configured to receive image data from an        imaging device, separating the image data into unmixed image        data, and providing the data through a plurality of data        channels, in which the unmixed image data in a first data        channel describes fluorescence signals attributable to a cell        nucleus, the unmixed data in a second data channel describes        fluorescence signals attributable to a subset tissue of        interest, and the unmixed data in a third data channel describes        fluorescence signals attributable to a first biomarker;    -   a cell masker configured to use image data from the first data        channel and generate an image representative of all nuclei in a        field of view and dilating the image to generate a cell mask;    -   a subset area masker configured to use image data from the        second data channel and generate a mask of all subset tissue        area in the field of view;    -   a first biomarker masker configured to use image data from the        third data channel and generate a first biomarker mask of all        cells in the field of view that express the first biomarker;    -   a subset cell masker configured to combine the cell mask and the        subset tissue area mask to generate at least one of a non-subset        cell mask and a subset cell mask;    -   an combination masker configured to combine the biomarker mask        and at least one of the non-subset cell mask and the subset cell        mask;    -   a positivity calculator configured to calculate a biomarker        positivity value.

Para. BO. The system of Para. BN in which the calculator is configuredto divide an area of the biomarker mask by an area of the at least oneof the non-subset cell mask and the subset cell mask to calculate thebiomarker positivity value.

Para. BP. The system of Para. BN in which the first biomarker comprisesa biomarker selected from PD-L1, PD-L2, B7-H3, B7-H4, HLA-DR, Galectin9, CD80, CD86, 4.1BBL, ICOSL, CD40, OX40L, IDO-1, GITRL, PD-1, TIM3,LAG3, 41BB, OX40, CTLA-4, CD40L, CD28, GITR, ICOS, CD28, CD3, CD4, CD8,FoxP3, CD25, CD16, CD56, CD68, CD163, CD80, and CD86.

Para. BQ. The system of Para. BN in which the second biomarker isdifferent from the first biomarker and comprises a biomarker selectedfrom PD-L1, PD-L2, B7-H3, B7-H4, HLA-DR, Galectin 9, CD80, CD86, 4.1BBL,ICOSL, CD40, OX40L, IDO-1, GITRL, PD-1, TIM3, LAG3, 41BB, OX40, CTLA-4,CD40L, CD28, GITR, ICOS, CD28, CD3, CD4, CD8, FoxP3, CD25, CD16, CD56,CD68, CD163, CD80, and CD86.

Para. BR. The system of Para. BN in which the subset tissue of interestis tumor tissue, the subset cell is a tumor cell, and the non-subsetcell is a non-tumor cell.

Para. BS. A system comprising an image processing unit configured to:

receive image data from an imaging device, the image data describing asample of tissue taken from a patient; and

determine a biomarker positivity value representative of an amount ofcells of a type in the sample expressing a biomarker of interestrelative to the total amount of cells of the type in the sample.

While certain embodiments have been illustrated and described, it shouldbe understood that changes and modifications can be made therein inaccordance with ordinary skill in the art without departing from thetechnology in its broader aspects as defined in the following claims.

The embodiments, illustratively described herein may suitably bepracticed in the absence of any element or elements, limitation orlimitations, not specifically disclosed herein. Thus, for example, theterms “comprising,” “including,” “containing,” etc. shall be readexpansively and without limitation. Additionally, the terms andexpressions employed herein have been used as terms of description andnot of limitation, and there is no intention in the use of such termsand expressions of excluding any equivalents of the features shown anddescribed or portions thereof, but it is recognized that variousmodifications are possible within the scope of the claimed technology.Additionally, the phrase “consisting essentially of” will be understoodto include those elements specifically recited and those additionalelements that do not materially affect the basic and novelcharacteristics of the claimed technology. The phrase “consisting of”excludes any element not specified.

The present disclosure is not to be limited in terms of the particularembodiments described in this application. Many modifications andvariations can be made without departing from its spirit and scope, aswill be apparent to those skilled in the art. Functionally equivalentmethods and compositions within the scope of the disclosure, in additionto those enumerated herein, will be apparent to those skilled in the artfrom the foregoing descriptions. Such modifications and variations areintended to fall within the scope of the appended claims. The presentdisclosure is to be limited only by the terms of the appended claims,along with the full scope of equivalents to which such claims areentitled. It is to be understood that this disclosure is not limited toparticular methods, reagents, compounds compositions or biologicalsystems, which can of course vary. It is also to be understood that theterminology used herein is for the purpose of describing particularembodiments only, and is not intended to be limiting.

In addition, where features or aspects of the disclosure are describedin terms of Markush groups, those skilled in the art will recognize thatthe disclosure is also thereby described in terms of any individualmember or subgroup of members of the Markush group.

As will be understood by one skilled in the art, for any and allpurposes, particularly in terms of providing a written description, allranges disclosed herein also encompass any and all possible subrangesand combinations of subranges thereof. Any listed range can be easilyrecognized as sufficiently describing and enabling the same range beingbroken down into at least equal halves, thirds, quarters, fifths,tenths, etc. As a non-limiting example, each range discussed herein canbe readily broken down into a lower third, middle third and upper third,etc. As will also be understood by one skilled in the art all languagesuch as “up to,” “at least,” “greater than,” “less than,” and the like,include the number recited and refer to ranges which can be subsequentlybroken down into subranges as discussed above. Finally, as will beunderstood by one skilled in the art, a range includes each individualmember.

All publications, patent applications, issued patents, and otherdocuments referred to in this specification are herein incorporated byreference as if each individual publication, patent application, issuedpatent, or other document was specifically and individually indicated tobe incorporated by reference in its entirety. Definitions that arecontained in text incorporated by reference are excluded to the extentthat they contradict definitions in this disclosure.

Other embodiments are set forth in the following claims.

What is claimed is:
 1. A system for determining a score representativeof a proximity between at least one pair of cells in a tumor tissuesample, the system comprising: a programmable controller programmed to:receive image data from an imaging device, separate the image data intounmixed image data, and provide the data through a plurality of datachannels; wherein the unmixed image data in a first data channeldescribes fluorescence signals attributable to a cell nucleus, theunmixed data in a second data channel describes fluorescence signalsattributable to tumor tissue, the unmixed data in a third data channeldescribes fluorescence signals attributable to a first biomarker, andthe unmixed image data in a fourth data channel describes fluorescencesignals attributable to a second biomarker; receive image data from thefirst data channel and generate an image representative of all nuclei ina field of view and dilate the image to generate a cell mask; receiveimage data from the second data channel and generate a mask of all tumorarea in the field of view; receive image data from the third datachannel and generate a first biomarker mask of all cells in the field ofview that express the first biomarker; receive image data from thefourth data channel and generate a second biomarker mask of all cells inthe field of view that express the second biomarker; combine the cellmask and the tumor area mask to generate at least one of a non-tumorcell mask and a tumor cell mask; dilate the first biomarker mask by amargin sufficient to encompass proximally located cells expressing thesecond biomarker to generate a dilated first biomarker mask; combine thedilated first biomarker mask and the second biomarker mask to generatean interaction mask; calculate a spatial proximity score according tothe equation: ${SPS} = {\frac{A_{I}}{A_{C}} \times 10^{4}}$ whereinA_(I) is a total interaction area (area of the interaction mask) andA_(C) is the total area of the at least one of the non-tumor cell mask,the tumor cell mask, and the cell mask.
 2. The system of claim 1 inwhich the programmable controller is further programmed to divide thearea of the interaction mask by the area of one of the non-tumor cellmask, the tumor cell mask, and the cell mask.
 3. The system of claim 1in which the margin sufficient to encompass proximally located cellsexpressing the second biomarker ranges from about 1 to about 100 pixels.4. The system of claim 1 in which a first member of the at least onepair of cells comprises a tumor cell and a second member of the at leastone pair of cells comprises a non-tumor cell.
 5. The system of claim 1in which each mask is generated by assigning a binary value to eachpixel of image data from a selected channel.
 6. The system of claim 5 inwhich the binary value is assigned to each value by a thresholdfunction, wherein the threshold function assigns a value of 1 to eachpixel that has an intensity equal to or greater than a predeterminedintensity threshold, and assigns a value of 0 to each pixel that has anintensity below the predetermined intensity threshold.
 7. The system ofclaim 5 in which the binary value is assigned to each value by ahistogram threshold function, wherein the histogram threshold functionuses a sliding scale of pixel intensity to determine a threshold, andassigns a value of 1 to each pixel that has an intensity equal to orgreater than an intensity threshold, and assigns a value of 0 to eachpixel that has an intensity below the intensity threshold.
 8. The systemof claim 7 in which the threshold is determined by: summing an intensityof every pixel into a total intensity; multiplying a thresholdpercentage by the total intensity to obtain a cut-off sum; grouping allpixels by intensity in a histogram; and summing the pixel intensitiesfrom lowest to highest until the cut-off sum is achieved; wherein thelast highest pixel intensity visited in the process is the intensitythreshold.
 9. The system of claim 1 in which combining the masks isperformed using an “and” operation; wherein the “and” operation is amultiplication of the pixel intensity of a pixel in a first mask by thepixel intensity of the pixel in a second mask.
 10. The system of claim 1in which combining the masks is performed using an “out” operation;wherein the “out” operation removes a second mask from a first mask. 11.A system for determining a value of biomarker positivity in a tumortissue sample, the system comprising: a programmable controllerprogrammed to: receive image data from an imaging device, separate theimage data into unmixed image data, and provide the data through aplurality of data channels, in which the unmixed image data in a firstdata channel describes fluorescence signals attributable to a cellnucleus, the unmixed data in a second data channel describesfluorescence signals attributable to a subset tissue of interest, andthe unmixed data in a third data channel describes fluorescence signalsattributable to a first biomarker; use image data from the first datachannel and generate an image representative of all nuclei in a field ofview and dilate the image to generate a cell mask; use image data fromthe second data channel and generate a mask of all subset tissue area inthe field of view; use image data from the third data channel andgenerate a first biomarker mask of all cells in the field of view thatexpress the first biomarker; combine the cell mask and the subset tissuearea mask to generate at least one of a non-subset cell mask and asubset cell mask; combine the biomarker mask and the non-subset cellmask or the subset cell mask to generate a combined mask of non-subsetcells or subset cells that are positive for the first biomarker; andcalculate a biomarker positivity value by either (i) dividing an area ofthe combined mask by an area of the non-subset cell mask or the subsetcell mask, or (ii) dividing an area of the first biomarker mask by anarea of the cell mask.
 12. The system of claim 11 in which the subsettissue of interest is tumor tissue, the subset cell is a tumor cell, andthe non-subset cell is a non-tumor cell.
 13. The system of claim 11 inwhich each mask is generated by assigning a binary value to each pixelof image data from a selected channel.
 14. The system of claim 13 inwhich the binary value is assigned to each value by a thresholdfunction, wherein the threshold function assigns a value of 1 to eachpixel that has an intensity equal to or greater than a predeterminedintensity threshold, and assigns a value of 0 to each pixel that has anintensity below the predetermined intensity threshold.
 15. The system ofclaim 13 in which the binary value is assigned to each value by ahistogram threshold function, wherein the histogram threshold functionuses a sliding scale of pixel intensity to determine a threshold, andassigns a value of 1 to each pixel that has an intensity equal to orgreater than an intensity threshold, and assigns a value of 0 to eachpixel that has an intensity below the intensity threshold.
 16. Thesystem of claim 11 in which combining the masks is performed using an“and” operation; wherein the “and” operation is a multiplication of thepixel intensity of a pixel in a first mask by the pixel intensity of apixel in a second mask.
 17. The system of claim 11 in which combiningthe masks is performed using an “out” operation; wherein the “out”operation removes a second mask from a first mask.